-- Logs begin at Fri 2022-10-14 17:15:39 UTC, end at Fri 2022-10-14 18:13:17 UTC. -- Oct 14 17:41:13.422514 dv-cinder-749155-15-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 14 17:41:13.430417 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 14 17:41:13 2022] *** Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: nodename: dv-cinder-749155-15-iscsi Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: machine: x86_64 Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: clock source: unix Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: pcre jit disabled Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: detected number of CPU cores: 4 Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: current working directory: / Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: detected binary path: /usr/bin/uwsgi-core Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: your processes number limit is 47680 Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: your memory page size is 4096 bytes Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: detected max file descriptor number: 2048 Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: lock engine: pthread robust mutexes Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: thunder lock: enabled Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 14 17:41:13.435914 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 14 17:41:13.474031 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: Python main interpreter initialized at 0x55c71be049c0 Oct 14 17:41:13.474031 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: python threads support enabled Oct 14 17:41:13.474031 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: your server socket listen backlog is limited to 100 connections Oct 14 17:41:13.474031 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: your mercy for graceful operations on workers is 90 seconds Oct 14 17:41:13.474417 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: mapped 403077 bytes (393 KB) for 2 cores Oct 14 17:41:13.474500 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: *** Operational MODE: preforking *** Oct 14 17:41:13.474715 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: *** uWSGI is running in multiple interpreter mode *** Oct 14 17:41:13.474715 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: spawned uWSGI master process (pid: 81252) Oct 14 17:41:13.474928 dv-cinder-749155-15-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 14 17:41:13.479392 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: spawned uWSGI worker 1 (pid: 81266, cores: 1) Oct 14 17:41:13.479392 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: spawned uWSGI worker 2 (pid: 81267, cores: 1) Oct 14 17:41:13.479392 dv-cinder-749155-15-iscsi devstack@keystone.service[81252]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 14 17:41:17.803479 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 17:41:17.803479 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: from cryptography.utils import int_from_bytes Oct 14 17:41:17.804878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 17:41:17.804878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: from cryptography.utils import int_from_bytes Oct 14 17:41:17.804998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 17:41:17.804998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: from cryptography.utils import int_from_bytes Oct 14 17:41:17.804998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 17:41:17.804998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: from cryptography.utils import int_from_bytes Oct 14 17:41:17.855748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 14 17:41:17.855963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 14 17:41:17.856058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 14 17:41:17.856150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] command line args: [] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 14 17:41:17.856223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 14 17:41:17.856357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] command line args: [] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 14 17:41:17.856487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 14 17:41:17.856559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ================================================================================ {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 14 17:41:17.856621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 14 17:41:17.856782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ================================================================================ {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 14 17:41:17.856870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] admin_token = **** {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.857031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] admin_token = **** {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.857128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] config_dir = [] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.857261 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.857346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] config_dir = [] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.857414 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] config_source = [] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.857474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.857634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] debug = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.857724 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] config_source = [] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.857791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] debug = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] list_limit = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] list_limit = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] log_config_append = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860895 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] log_config_append = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.860959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] log_dir = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] log_dir = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] log_file = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] log_file = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861758 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.861948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862016 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862276 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.862953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.863012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.863070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.863137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.863919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.864105 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.864342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] publish_errors = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.864509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] publish_errors = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.864717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.864947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.865122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.865355 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.865567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.865740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.865937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.866182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.866393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.866558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.866751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] standard_threads = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.867009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] standard_threads = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.867206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.867437 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.867643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.867811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.868011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.868217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.868412 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] use_journal = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.868690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] use_journal = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.868891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] use_json = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.869061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] use_json = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.869225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] use_stderr = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.869358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] use_stderr = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.869358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] use_syslog = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.869670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] use_syslog = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.869832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.869971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 17:41:17.870109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.870238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.870367 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.870552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.870716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.870851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.870987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.871115 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.871245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.871381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.871511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.871639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.871767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.872090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.872234 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.872376 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.872529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.872681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.872813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.872941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.873069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.873206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.873387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.873522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.873651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.873779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.873913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.874054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.874184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.874320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.874460 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.874590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.874726 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.874864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.875000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.875134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.875269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.875399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.875600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.875732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.875861 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.876019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.876148 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.876297 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.876436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.876566 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.876695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.876822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.876965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.877095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.877232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.877361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.877497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.877626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.877754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.877883 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.878019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.878158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.878294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.878423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.878552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.878680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.878816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.878945 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.879085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.879214 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.879342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.879470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.879607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.879736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.879890 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.880028 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.880158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.880286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.880429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.880564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.880695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.880822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.880950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.881078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.881213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.881349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.881478 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.881605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.881744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.881915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.882127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.882333 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.882486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.882624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.882760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.882891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.883031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.883161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.883296 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.883467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.883608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.885038 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.885283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.885697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.885930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.886100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.886294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.886487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.886718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.886885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.887111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.887278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.887472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.887664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.888001 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.888184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.888417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.888655 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.888851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.889080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.889249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.889442 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.889666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.889835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.890029 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.890230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.890393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.890558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.890737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.890881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.891021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.891198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.891331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.891473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.891605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.891740 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.891920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.892058 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.892190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.892329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.892462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.892612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.892744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.892874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.893003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.893176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.893313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.893443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.893612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.893751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.893883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.894021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.894153 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.894284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.894414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.894553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.894684 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.894823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.894954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.895084 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.895220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.895351 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.895481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.895618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.895785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.895964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.896103 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.896233 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.896363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.896514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.896647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.896776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.896906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.897041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.897172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.897310 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.897935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.898070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.898200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.898337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.898469 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.898607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.898749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.898880 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.899009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.899139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.899275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.899478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.899612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.899742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.899871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.900025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.900171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.900311 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.900450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.900583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.900725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.900856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.900986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.901115 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.901262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.901394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.901524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.901654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.901790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.901921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.902059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.902189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.902320 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.902450 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.902591 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.902722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.902864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.903003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.903134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.903270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.903401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.903557 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.903738 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] auth.external = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.903929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] auth.external = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.904081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.904222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.904361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.904506 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.904640 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.904772 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.904911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] auth.password = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.905044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] auth.password = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.905175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] auth.token = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.905306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] auth.token = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.905436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.905573 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.905714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.905934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.906084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.906217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.906349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.906491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.906637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.906770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.906903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.907034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.907164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.907294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.907434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] credential.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.907567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] credential.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.907704 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.907837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.907992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.908125 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.908279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.908413 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.908546 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.908677 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.908807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.908940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.909079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.909211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.909342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.909473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.909604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.909780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.909925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.910058 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.910205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.910337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.910468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.910605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.910746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.910920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.911063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.911195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.911331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.911469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.911609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.911741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.911981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] federation.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.912124 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] federation.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.912276 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.912415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.912556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.912695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.912833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.913005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.913809 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.913976 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.914116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.914251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.914400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.914593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.914593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.914870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.915005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.915137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.915269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.915400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.915531 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.915670 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.915808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.915980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.916123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.916267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.916399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.916620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.916766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.916899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.917030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.917161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.917294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.917434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.917566 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.917697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.917828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.917959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.917959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.918213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.918348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.918478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.918620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.918751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.918882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.919020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.919152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.919283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.919415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.919545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.919676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.919815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.920042 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.920187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.920320 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.920451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.920593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.920737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.920870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.921002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.921134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.921264 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.921395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.921534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.921667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.921798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.921929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.922060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.922191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.922339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.922472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.922604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.922735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.922866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.922997 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.923135 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.923267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.923398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.923530 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.923662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.923662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.923986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.924119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.924260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.924391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.924521 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.924664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.924796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.924927 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.925065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.925196 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.925326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.925464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.925596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.925728 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.925859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.926006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.926148 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.926287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.926419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.926549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.926678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.926809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.926940 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.927079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.927210 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.927341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.927471 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.927600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.927730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.927933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.928073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.928204 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.928334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.928465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.928595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.928846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.929059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.929268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.929421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.929566 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.929699 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.929838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.929971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.930112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.930244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.930374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.930504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.930644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.930776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.930907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.931038 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.931168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.931298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.931436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.931568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.931706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.931848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.932005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.932136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.932279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.932412 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.932543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.932673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.932673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.932918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.933058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.933190 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.933190 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.933435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.933565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.933705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.933845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.934018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.934155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.934285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.934415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.934545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.934683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.934816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.934947 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.935078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.935209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.935339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.935487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.935625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.935758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.935937 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.936083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.936216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.936358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.936490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.936620 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.936750 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.936880 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.937010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.937155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.937288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.937430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.937561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.937691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.937821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.937960 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.938091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.938222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.938354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.938485 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.938614 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.938752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.938883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.939014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.939144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.939286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.939417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.939555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.939686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.939817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.939968 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.940100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.940230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.940373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] resource.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.940373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.940620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] resource.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.940749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.940878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.941008 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.941160 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.941292 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.941425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.941555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.941685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.941822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.941954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.942083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.942213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.942342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.942474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.942687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.942820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.942960 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] role.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.943091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.943220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] role.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.943350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] role.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.943488 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.943621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] role.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.943751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.943966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.944138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.944315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.944503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.944639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.944771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.944941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.945122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.945256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.945405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.945538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.945670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.945801 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.945931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.946073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.946213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.946346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.946476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.946607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.946738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.946879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.947019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.947152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.947283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.947414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.947414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.947660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.947660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.947955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.948095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.948228 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.948228 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.948473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.948625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.948760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.948904 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.949050 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.949185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.949325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.949459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.949590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.949720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.949852 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.949983 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.950122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.950254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.950395 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.950527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.950658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.950787 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.950935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.951067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.951199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.951336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.951474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.951611 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.951755 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.951960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.952107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.952244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.952392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.952529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.952678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.952818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.952955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] token.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.953092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] token.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.953228 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.953363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.953508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.953646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.953783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.953920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.954057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.954247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.954400 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.954563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.954702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.954840 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.954976 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.955114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.955259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.955397 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.955534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.955671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.955807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.956001 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.956156 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.956308 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.956445 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.956581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.956717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.956852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.956996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.957136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.957273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.957418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.957570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.957720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.957859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.957995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.958131 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.connection = **** {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.958286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.connection = **** {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.958425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.958562 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.958698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.958835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.958971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.959116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.959255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.959392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.959528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.959665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.959801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.960489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.960628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.960771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.960902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.961032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.961198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.961344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.961477 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.961608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.961738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.961868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.961998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.962137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.962269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.962401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.962543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.962674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.962805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.962944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.963076 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.963205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.963336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.963467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.963597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.963735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 17:41:17.963867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 14 17:41:17.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 14 17:41:17.972725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:17.972959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:17.973186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:17.973542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:17.973775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:17.973994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:17.982347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:17.982902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.011131 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.011413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.011561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.011716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.065405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.065876 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.114780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.115068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.115669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.115923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.273301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.273633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.273787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.273925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.273925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.274213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.274348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.274480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.274612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.274834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.275012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.275150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.297481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.297753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.297905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.298060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.298216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.298356 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.298493 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.298640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.298778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.298925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.299066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.299204 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.299341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.299490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.310092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.310284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.323347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.323552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.323693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.323843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.324020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.324162 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.324299 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.324435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.324585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.324737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.324876 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.325013 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.325151 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.325151 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 17:41:18.342014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.342203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 17:41:18.342547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.342694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.342832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 17:41:18.342982 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 17:41:18.343135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.343275 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.343423 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.343562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 17:41:18.343698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.343845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.348608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.349351 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.349351 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.349351 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.353422 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.353707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.353975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.354158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.354306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.354461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.354704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.354990 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.361194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.362057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.364086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.364370 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.364651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.364932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.365081 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.365230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.365382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.365521 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.365658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.365795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.365932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.366090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.366262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.366439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.366789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.366940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.367154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.367305 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.367444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.367606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.367868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.368167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.390467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.392225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.392397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.392508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.393526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.393846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.396989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.397250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.397588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.399373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.399664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.399973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.404424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.404715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.407857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.407857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.410519 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.410813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.411093 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.411373 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.411646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.411940 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.412205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.412619 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.412760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.412936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.413236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.413528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.413995 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.414420 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.414837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.415253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.428253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.428529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.430881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.431166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.432570 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.434991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.435749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.436060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.438212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.438514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.440392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.442684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.443584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.446427 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.447465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.447756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.448066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.448346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.448622 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.448921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.449847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.450155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.450444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.450737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.451023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.451317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.464934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.465212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.465477 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.465737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.467298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.467578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.467845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.468274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.472556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.474097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.475281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.476154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.476594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.477371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.478921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.479199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.479339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.479506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.479797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.480114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.480508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.481883 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.482172 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.482468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.482751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.483157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.492149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.492436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.492719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.493002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.493377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.493675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.493950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.494441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.494741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.495019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.495304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.495686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.496026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.496313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.509816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.512641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.512971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.515545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.515808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.517266 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.518472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.518714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.518995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.519420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.520043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.521525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.521815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.522220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.526078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.527527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.528709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.530179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.530394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.530795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.531078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.531351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.533094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.533494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.533790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.534072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.539826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.540151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.540432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.542448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.542744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.543029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.547240 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.547525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.547803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.548116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.548390 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.548655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.548928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.549202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.549486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.549760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.549889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 17:41:18.550115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.550251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.550379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.550507 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.550628 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.550757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.550876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.550994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.551260 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.551539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.551796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.552134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.552423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.552773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.553047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.553313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.553586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 17:41:18.586136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c71be049c0 pid: 81267 (default app) Oct 14 17:41:18.588903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c71be049c0 pid: 81266 (default app) Oct 14 17:41:18.589448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a2241-d370-494b-b80e-43ac7e5a393d None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:18.589728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a2241-d370-494b-b80e-43ac7e5a393d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:18.589990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a2241-d370-494b-b80e-43ac7e5a393d None None] PATH_INFO: `/v3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:18.594343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Fri Oct 14 17:41:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:25.446704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5885269-85bf-4827-acd5-d094c2cefbf7 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:25.447205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5885269-85bf-4827-acd5-d094c2cefbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:25.447310 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5885269-85bf-4827-acd5-d094c2cefbf7 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:25.451504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:25.456838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:25.457274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:25.457416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:25.528568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: INFO dbcounter [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] Registered counter for database keystone Oct 14 17:41:25.548323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG oslo_db.sqlalchemy.engines [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81267) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 14 17:41:25.554572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writer thread running {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 14 17:41:25.741927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG passlib.handlers.bcrypt [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81267) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 14 17:41:25.742228 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG passlib.handlers.bcrypt [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81267) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 14 17:41:25.801935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:25.803652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:25.933894 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 14 17:41:26.390063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5885269-85bf-4827-acd5-d094c2cefbf7 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:26.462497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: INFO dbcounter [None req-d5885269-85bf-4827-acd5-d094c2cefbf7 None None] Registered counter for database keystone Oct 14 17:41:26.484375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG oslo_db.sqlalchemy.engines [None req-d5885269-85bf-4827-acd5-d094c2cefbf7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81266) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 14 17:41:26.490667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writer thread running {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 14 17:41:26.610512 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5885269-85bf-4827-acd5-d094c2cefbf7 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:26.614643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a866ae0-02c0-46ce-84be-3276b236912e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:26.615255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a866ae0-02c0-46ce-84be-3276b236912e admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:26.615528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a866ae0-02c0-46ce-84be-3276b236912e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:26.615795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a866ae0-02c0-46ce-84be-3276b236912e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:26.625008 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a866ae0-02c0-46ce-84be-3276b236912e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:26.677871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6a866ae0-02c0-46ce-84be-3276b236912e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 14 17:41:26.678211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a866ae0-02c0-46ce-84be-3276b236912e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:26.688236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Fri Oct 14 17:41:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 17:41:26.694989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:26.712579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d47bb-a431-4892-971d-4823a736e88a None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:26.716956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c6250-9e03-4613-b79d-fff2bc5fa250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:26.718082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c6250-9e03-4613-b79d-fff2bc5fa250 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:26.718364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c6250-9e03-4613-b79d-fff2bc5fa250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:26.718637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c6250-9e03-4613-b79d-fff2bc5fa250 admin admin] PATH_INFO: `/v3/projects/a41cbe9ae3574988a4c391824f7f340a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:26.720262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c6250-9e03-4613-b79d-fff2bc5fa250 admin admin] RBAC: Authorizing `identity:get_project(project_id=a41cbe9ae3574988a4c391824f7f340a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:26.773834 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2b3c6250-9e03-4613-b79d-fff2bc5fa250 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 14 17:41:26.774175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c6250-9e03-4613-b79d-fff2bc5fa250 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:26.778779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Fri Oct 14 17:41:26 2022] GET /identity/v3/projects/a41cbe9ae3574988a4c391824f7f340a => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:41:29.036691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f3cfc-690f-45ec-afd5-5c6857c8fb82 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:29.037293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f3cfc-690f-45ec-afd5-5c6857c8fb82 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:29.037378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f3cfc-690f-45ec-afd5-5c6857c8fb82 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:29.043836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:29.049593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79eae92-3400-4a26-a352-37bf46cbd6b7 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:29.050026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79eae92-3400-4a26-a352-37bf46cbd6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:29.050136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79eae92-3400-4a26-a352-37bf46cbd6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:29.071737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-d79eae92-3400-4a26-a352-37bf46cbd6b7 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:29.073313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d79eae92-3400-4a26-a352-37bf46cbd6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:29.081280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 17:41:29.380234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f3cfc-690f-45ec-afd5-5c6857c8fb82 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:29.392666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f3cfc-690f-45ec-afd5-5c6857c8fb82 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:29.396802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00473b1a-95cc-40b5-8b4a-8cbb047eb20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:29.397932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00473b1a-95cc-40b5-8b4a-8cbb047eb20f admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:29.398205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00473b1a-95cc-40b5-8b4a-8cbb047eb20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:29.398475 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00473b1a-95cc-40b5-8b4a-8cbb047eb20f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:29.406143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00473b1a-95cc-40b5-8b4a-8cbb047eb20f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:29.407925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00473b1a-95cc-40b5-8b4a-8cbb047eb20f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:29.417319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Fri Oct 14 17:41:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:41:29.422832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79eae92-3400-4a26-a352-37bf46cbd6b7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:29.431897 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79eae92-3400-4a26-a352-37bf46cbd6b7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:29.436260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393fefe-f12b-48d9-852c-44278ad29558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:29.436882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393fefe-f12b-48d9-852c-44278ad29558 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:29.437159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393fefe-f12b-48d9-852c-44278ad29558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:29.437429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393fefe-f12b-48d9-852c-44278ad29558 admin admin] PATH_INFO: `/v3/users/92acc9f005ce4ccfaadbc688dbb479a3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:29.438723 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393fefe-f12b-48d9-852c-44278ad29558 admin admin] RBAC: Authorizing `identity:get_user(user_id=92acc9f005ce4ccfaadbc688dbb479a3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:29.440602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393fefe-f12b-48d9-852c-44278ad29558 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:29.442116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Fri Oct 14 17:41:29 2022] GET /identity/v3/users/92acc9f005ce4ccfaadbc688dbb479a3 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:31.623465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207539c-b092-4e7c-b5c8-bb669e209d04 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:31.624038 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207539c-b092-4e7c-b5c8-bb669e209d04 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:31.624038 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207539c-b092-4e7c-b5c8-bb669e209d04 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:31.625510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:41:31.631233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3680ad-3913-402d-b637-3fdbb413024b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:31.631835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3680ad-3913-402d-b637-3fdbb413024b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:31.631952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3680ad-3913-402d-b637-3fdbb413024b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:31.651702 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9c3680ad-3913-402d-b637-3fdbb413024b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:31.652941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9c3680ad-3913-402d-b637-3fdbb413024b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:31.734319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 6/11] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:31.746578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636af8-89f1-42f0-86c6-d15ef9595e27 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:31.746864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636af8-89f1-42f0-86c6-d15ef9595e27 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:31.747139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636af8-89f1-42f0-86c6-d15ef9595e27 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:31.748474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 6/12] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:31.753830 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb20072-7cce-4d2c-ad23-4c98c227e83b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:31.754117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb20072-7cce-4d2c-ad23-4c98c227e83b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:31.754391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb20072-7cce-4d2c-ad23-4c98c227e83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:31.774978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8fb20072-7cce-4d2c-ad23-4c98c227e83b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:31.776203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8fb20072-7cce-4d2c-ad23-4c98c227e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:31.792852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:31.848156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:41:32.005465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94636af8-89f1-42f0-86c6-d15ef9595e27 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:32.122153 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb20072-7cce-4d2c-ad23-4c98c227e83b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:32.152693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94636af8-89f1-42f0-86c6-d15ef9595e27 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:32.187951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b787981-590a-4820-8ff5-89cbf341dd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:32.188794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b787981-590a-4820-8ff5-89cbf341dd5a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:32.189070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b787981-590a-4820-8ff5-89cbf341dd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:32.189339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b787981-590a-4820-8ff5-89cbf341dd5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:32.201024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b787981-590a-4820-8ff5-89cbf341dd5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-3b787981-590a-4820-8ff5-89cbf341dd5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:32.209918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:32.210833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:41:32.211662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:41:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:32.216810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b787981-590a-4820-8ff5-89cbf341dd5a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:32.224161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb20072-7cce-4d2c-ad23-4c98c227e83b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:32.252152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:32.260029 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf5369-6f93-4710-a35c-e9ce68ceb245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:32.261035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf5369-6f93-4710-a35c-e9ce68ceb245 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:32.261150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf5369-6f93-4710-a35c-e9ce68ceb245 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:32.261368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf5369-6f93-4710-a35c-e9ce68ceb245 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:32.262042 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf5369-6f93-4710-a35c-e9ce68ceb245 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:32.263430 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf5369-6f93-4710-a35c-e9ce68ceb245 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:32.274331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.sql.core [None req-c7cf5369-6f93-4710-a35c-e9ce68ceb245 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 17:41:32.274331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 17:41:32.274331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [parameters: {'id': '26078969f75048f284ffdba8f38b536b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 17:41:32.274331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81267) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-c7cf5369-6f93-4710-a35c-e9ce68ceb245 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application conn.query(q) Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 17:41:32.292903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result.read() Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 14 17:41:32.293938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application next(self.gen) Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application yield resource Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application next(self.gen) Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise self.value Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application session.commit() Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 17:41:32.294880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.session.flush() Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application compat.raise_( Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 17:41:32.296283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 17:41:32.297145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application conn.query(q) Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result.read() Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 17:41:32.297907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [parameters: {'id': '26078969f75048f284ffdba8f38b536b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 14 17:41:32.299149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:32.300077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 8/15] 10.222.0.27 () {60 vars in 1137 bytes} [Fri Oct 14 17:41:32 2022] POST /identity/v3/domains => generated 190 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:41:32.301096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf5369-6f93-4710-a35c-e9ce68ceb245 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:32.320788 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:32.323199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b787981-590a-4820-8ff5-89cbf341dd5a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:32.357864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d7473-203b-48f8-b6e9-419379d20cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:32.358479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d7473-203b-48f8-b6e9-419379d20cad None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:32.358757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d7473-203b-48f8-b6e9-419379d20cad None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:32.359034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d7473-203b-48f8-b6e9-419379d20cad None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:32.359703 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d7473-203b-48f8-b6e9-419379d20cad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:32.360039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d7473-203b-48f8-b6e9-419379d20cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:32.361432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d7473-203b-48f8-b6e9-419379d20cad None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:32.373391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 8/16] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:41:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:41:32.378896 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d7473-203b-48f8-b6e9-419379d20cad None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:32.410390 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf5369-6f93-4710-a35c-e9ce68ceb245 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:32.445635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115b2ae-3115-4f84-a031-640a71fb27c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:32.446261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115b2ae-3115-4f84-a031-640a71fb27c3 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:32.446535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115b2ae-3115-4f84-a031-640a71fb27c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:32.446802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115b2ae-3115-4f84-a031-640a71fb27c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:32.456685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115b2ae-3115-4f84-a031-640a71fb27c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:32.458167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115b2ae-3115-4f84-a031-640a71fb27c3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:32.473433 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-6115b2ae-3115-4f84-a031-640a71fb27c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:41:32.474451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:32.475324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:41:32.476170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:41:32.476170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:41:32.476170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:32.476170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:32.481441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115b2ae-3115-4f84-a031-640a71fb27c3 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:32.485179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d7473-203b-48f8-b6e9-419379d20cad None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:32.518952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a7057-ff51-45fa-8721-eadfa0d2dba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:32.519511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a7057-ff51-45fa-8721-eadfa0d2dba2 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:32.519781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a7057-ff51-45fa-8721-eadfa0d2dba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:32.520082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a7057-ff51-45fa-8721-eadfa0d2dba2 None admin] PATH_INFO: `/v3/users/92acc9f005ce4ccfaadbc688dbb479a3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:32.521633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a7057-ff51-45fa-8721-eadfa0d2dba2 None admin] RBAC: Authorizing `identity:get_user(user_id=92acc9f005ce4ccfaadbc688dbb479a3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:32.523667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a7057-ff51-45fa-8721-eadfa0d2dba2 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:32.525251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 9/18] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:32 2022] GET /identity/v3/users/92acc9f005ce4ccfaadbc688dbb479a3 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:41:32.530669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a7057-ff51-45fa-8721-eadfa0d2dba2 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:32.550959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:32.585927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115b2ae-3115-4f84-a031-640a71fb27c3 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:32.620415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d0264-7cb8-48af-bea4-c1f0f066b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:32.621012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d0264-7cb8-48af-bea4-c1f0f066b2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:32.621290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d0264-7cb8-48af-bea4-c1f0f066b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:32.621563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d0264-7cb8-48af-bea4-c1f0f066b2b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:32.622201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d0264-7cb8-48af-bea4-c1f0f066b2b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:32.622503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d0264-7cb8-48af-bea4-c1f0f066b2b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:32.623911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d0264-7cb8-48af-bea4-c1f0f066b2b4 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:32.629636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a7057-ff51-45fa-8721-eadfa0d2dba2 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:32.641334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 10/19] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:41:32.665866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48582733-ed1a-4674-83cb-bd699a310e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:32.666526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48582733-ed1a-4674-83cb-bd699a310e81 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:32.666806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48582733-ed1a-4674-83cb-bd699a310e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:32.667079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48582733-ed1a-4674-83cb-bd699a310e81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:32.668400 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48582733-ed1a-4674-83cb-bd699a310e81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:32.669903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48582733-ed1a-4674-83cb-bd699a310e81 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:32.671548 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 10/20] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:41:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:41:32.677487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d0264-7cb8-48af-bea4-c1f0f066b2b4 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:32.783233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d0264-7cb8-48af-bea4-c1f0f066b2b4 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:32.811544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:32.820601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e31ca-43fa-4134-afc9-27d38d9bd856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:32.821389 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e31ca-43fa-4134-afc9-27d38d9bd856 None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:32.821578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e31ca-43fa-4134-afc9-27d38d9bd856 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:32.821854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e31ca-43fa-4134-afc9-27d38d9bd856 None admin] PATH_INFO: `/v3/domains/default/users/92acc9f005ce4ccfaadbc688dbb479a3/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:32.823908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e31ca-43fa-4134-afc9-27d38d9bd856 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=92acc9f005ce4ccfaadbc688dbb479a3, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:32.825352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e31ca-43fa-4134-afc9-27d38d9bd856 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:32.940776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 11/21] 10.222.0.27 () {58 vars in 1361 bytes} [Fri Oct 14 17:41:32 2022] PUT /identity/v3/domains/default/users/92acc9f005ce4ccfaadbc688dbb479a3/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:41:35.176182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a7509-2b5d-4159-ac2e-229092aacb5f None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:35.176695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a7509-2b5d-4159-ac2e-229092aacb5f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:35.176695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a7509-2b5d-4159-ac2e-229092aacb5f None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:35.178252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 11/22] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:35.184740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f1476-3155-45cd-bcc7-fd0460134b74 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:35.185158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f1476-3155-45cd-bcc7-fd0460134b74 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:35.185348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f1476-3155-45cd-bcc7-fd0460134b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:35.206331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-021f1476-3155-45cd-bcc7-fd0460134b74 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:35.207851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-021f1476-3155-45cd-bcc7-fd0460134b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:35.304764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 12/23] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:35.441946 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3dda-0f2f-440f-a61d-b6a72233848c None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:35.442236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3dda-0f2f-440f-a61d-b6a72233848c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:35.442419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3dda-0f2f-440f-a61d-b6a72233848c None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:35.444479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 12/24] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:35.450551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fce012-876e-4198-b6df-53779a253628 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:35.450840 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fce012-876e-4198-b6df-53779a253628 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:35.451175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fce012-876e-4198-b6df-53779a253628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:35.480240 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f5fce012-876e-4198-b6df-53779a253628 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:35.481530 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f5fce012-876e-4198-b6df-53779a253628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:35.578892 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 13/25] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:35.611111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b8da-a000-412d-8670-a8770179389e None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:35.611414 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b8da-a000-412d-8670-a8770179389e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:35.611654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b8da-a000-412d-8670-a8770179389e None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:35.613196 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 13/26] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:35.618290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ede096-e543-4f91-8be3-62c0f31c4cf9 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:35.618582 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ede096-e543-4f91-8be3-62c0f31c4cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:35.618854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ede096-e543-4f91-8be3-62c0f31c4cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:35.642983 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-74ede096-e543-4f91-8be3-62c0f31c4cf9 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:35.644181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-74ede096-e543-4f91-8be3-62c0f31c4cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:35.704789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:35.735642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 14/27] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:35.858729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601b8da-a000-412d-8670-a8770179389e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:35.898545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:35.966218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ede096-e543-4f91-8be3-62c0f31c4cf9 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:35.998972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601b8da-a000-412d-8670-a8770179389e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:36.080590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e497298-e82d-41ba-b461-e3eb950a7a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:36.081230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e497298-e82d-41ba-b461-e3eb950a7a3d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:36.081505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e497298-e82d-41ba-b461-e3eb950a7a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:36.081775 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e497298-e82d-41ba-b461-e3eb950a7a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:36.115341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e497298-e82d-41ba-b461-e3eb950a7a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:36.117517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e497298-e82d-41ba-b461-e3eb950a7a3d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:36.119756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ede096-e543-4f91-8be3-62c0f31c4cf9 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-7e497298-e82d-41ba-b461-e3eb950a7a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:41:36.152045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:36.152943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:41:36.153763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:41:36.153763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:41:36.153763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:41:36.153763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 14/28] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:36.154732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528edd8-4aea-4ef0-82bc-56fe1abfad29 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:36.154829 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528edd8-4aea-4ef0-82bc-56fe1abfad29 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:36.155300 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528edd8-4aea-4ef0-82bc-56fe1abfad29 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:36.156566 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 15/29] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:36.162598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68b4db-87f0-463d-a2e5-8b62c8977242 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:36.163563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68b4db-87f0-463d-a2e5-8b62c8977242 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:36.163796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68b4db-87f0-463d-a2e5-8b62c8977242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:36.188303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2991d-bdf0-4005-8a3b-92a205800047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:36.188933 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2991d-bdf0-4005-8a3b-92a205800047 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:36.189208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2991d-bdf0-4005-8a3b-92a205800047 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:36.189537 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2991d-bdf0-4005-8a3b-92a205800047 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:36.195941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG passlib.handlers.bcrypt [None req-ac68b4db-87f0-463d-a2e5-8b62c8977242 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81266) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 14 17:41:36.196860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG passlib.handlers.bcrypt [None req-ac68b4db-87f0-463d-a2e5-8b62c8977242 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81266) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 14 17:41:36.203060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2991d-bdf0-4005-8a3b-92a205800047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-69a2991d-bdf0-4005-8a3b-92a205800047 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:36.210328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:36.211305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:41:36.212094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:36.214636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 15/30] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:41:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:36.215591 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2991d-bdf0-4005-8a3b-92a205800047 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:36.257660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ac68b4db-87f0-463d-a2e5-8b62c8977242 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:36.259224 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ac68b4db-87f0-463d-a2e5-8b62c8977242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:36.262104 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:36.337466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2991d-bdf0-4005-8a3b-92a205800047 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:36.363123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 16/31] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:36.364329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68b4db-87f0-463d-a2e5-8b62c8977242 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:36.383161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:36.409005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecd553-2635-4a17-9f8e-7a64e3e1dbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:36.410173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecd553-2635-4a17-9f8e-7a64e3e1dbbd None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:36.411446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecd553-2635-4a17-9f8e-7a64e3e1dbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:36.414894 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecd553-2635-4a17-9f8e-7a64e3e1dbbd None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:36.415802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecd553-2635-4a17-9f8e-7a64e3e1dbbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:36.416357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecd553-2635-4a17-9f8e-7a64e3e1dbbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:36.418361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecd553-2635-4a17-9f8e-7a64e3e1dbbd None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:36.449747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 16/32] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:36.455036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecd553-2635-4a17-9f8e-7a64e3e1dbbd None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:36.566114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68b4db-87f0-463d-a2e5-8b62c8977242 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:36.609791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:36.615111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecd553-2635-4a17-9f8e-7a64e3e1dbbd None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:36.652080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151eacdf-231e-422b-81e6-5a81586ccf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:36.653298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151eacdf-231e-422b-81e6-5a81586ccf97 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:36.653768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151eacdf-231e-422b-81e6-5a81586ccf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:36.654252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151eacdf-231e-422b-81e6-5a81586ccf97 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:36.655289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151eacdf-231e-422b-81e6-5a81586ccf97 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:36.657038 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151eacdf-231e-422b-81e6-5a81586ccf97 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:36.685741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d4643-a4d2-4faa-81b2-0afc393cca94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:36.686715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d4643-a4d2-4faa-81b2-0afc393cca94 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:36.687186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d4643-a4d2-4faa-81b2-0afc393cca94 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:36.687677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d4643-a4d2-4faa-81b2-0afc393cca94 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:36.688606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d4643-a4d2-4faa-81b2-0afc393cca94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:36.689098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d4643-a4d2-4faa-81b2-0afc393cca94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:36.690690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d4643-a4d2-4faa-81b2-0afc393cca94 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:36.704749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 17/33] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:41:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:36.705957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d4643-a4d2-4faa-81b2-0afc393cca94 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:36.804985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 17/34] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Oct 14 17:41:36 2022] POST /identity/v3/roles => generated 225 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:41:36.805338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151eacdf-231e-422b-81e6-5a81586ccf97 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:36.820565 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d4643-a4d2-4faa-81b2-0afc393cca94 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:36.895230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323b4d3-83b1-4e64-b7cd-50f2b59e7c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:36.899623 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323b4d3-83b1-4e64-b7cd-50f2b59e7c12 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:36.900265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323b4d3-83b1-4e64-b7cd-50f2b59e7c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:36.902427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323b4d3-83b1-4e64-b7cd-50f2b59e7c12 None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:36.906218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323b4d3-83b1-4e64-b7cd-50f2b59e7c12 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:36.908218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323b4d3-83b1-4e64-b7cd-50f2b59e7c12 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:36.968406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 18/35] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:41:36 2022] POST /identity/v3/projects => generated 301 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:41:36.970422 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323b4d3-83b1-4e64-b7cd-50f2b59e7c12 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:37.001646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:37.048385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151eacdf-231e-422b-81e6-5a81586ccf97 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:37.097562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcea4eb-d770-4ccd-81c0-2ce488cce71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:37.098585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcea4eb-d770-4ccd-81c0-2ce488cce71b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:37.099078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcea4eb-d770-4ccd-81c0-2ce488cce71b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:37.099644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcea4eb-d770-4ccd-81c0-2ce488cce71b None admin] PATH_INFO: `/v3/users/92acc9f005ce4ccfaadbc688dbb479a3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:37.102502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcea4eb-d770-4ccd-81c0-2ce488cce71b None admin] RBAC: Authorizing `identity:get_user(user_id=92acc9f005ce4ccfaadbc688dbb479a3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:37.106525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcea4eb-d770-4ccd-81c0-2ce488cce71b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:37.109184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 18/36] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:36 2022] GET /identity/v3/users/92acc9f005ce4ccfaadbc688dbb479a3 => generated 247 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:37.110918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc2d34-6a3b-4745-830a-d197af83f0fc None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:37.111507 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc2d34-6a3b-4745-830a-d197af83f0fc None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:37.112021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc2d34-6a3b-4745-830a-d197af83f0fc None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:37.113604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 19/37] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:37.115524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc2d34-6a3b-4745-830a-d197af83f0fc None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:37.135238 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:37.238561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323b4d3-83b1-4e64-b7cd-50f2b59e7c12 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:37.254460 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc2d34-6a3b-4745-830a-d197af83f0fc None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:37.276420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbff835-888e-438b-995a-a927f4bb5ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:37.277032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbff835-888e-438b-995a-a927f4bb5ec5 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:37.277310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbff835-888e-438b-995a-a927f4bb5ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:37.277574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbff835-888e-438b-995a-a927f4bb5ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:37.278232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbff835-888e-438b-995a-a927f4bb5ec5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:37.279609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbff835-888e-438b-995a-a927f4bb5ec5 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:37.290092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935989b-b0b3-4248-8a4f-fad25b17a705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:37.290898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935989b-b0b3-4248-8a4f-fad25b17a705 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:37.291284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935989b-b0b3-4248-8a4f-fad25b17a705 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:37.291653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935989b-b0b3-4248-8a4f-fad25b17a705 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:37.293060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935989b-b0b3-4248-8a4f-fad25b17a705 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:37.294597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935989b-b0b3-4248-8a4f-fad25b17a705 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:37.294732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 19/38] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Oct 14 17:41:36 2022] POST /identity/v3/roles => generated 219 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:41:37.295926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94413c-1e81-477f-9677-d09f3480c9e1 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:37.296219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94413c-1e81-477f-9677-d09f3480c9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:37.296578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94413c-1e81-477f-9677-d09f3480c9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:37.302128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 20/39] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:41:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:37.302128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ba000-8890-43c0-826c-84a1bad7242b None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:37.302128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ba000-8890-43c0-826c-84a1bad7242b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:37.302128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ba000-8890-43c0-826c-84a1bad7242b None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:37.302489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 21/40] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:37.303570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ba000-8890-43c0-826c-84a1bad7242b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:37.324254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-bb94413c-1e81-477f-9677-d09f3480c9e1 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:37.324785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:37.325504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-bb94413c-1e81-477f-9677-d09f3480c9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:37.423906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 20/41] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:37.423906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88059f05-182b-4c43-8a9c-3e9481579c6e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:37.423906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88059f05-182b-4c43-8a9c-3e9481579c6e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:37.423906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88059f05-182b-4c43-8a9c-3e9481579c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:37.430230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:37.450962 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ba000-8890-43c0-826c-84a1bad7242b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:37.455476 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-88059f05-182b-4c43-8a9c-3e9481579c6e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:37.456999 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-88059f05-182b-4c43-8a9c-3e9481579c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:37.497540 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e55e8e-17fd-496c-bdc5-55e6cd3e5e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:37.498460 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e55e8e-17fd-496c-bdc5-55e6cd3e5e4b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:37.498743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e55e8e-17fd-496c-bdc5-55e6cd3e5e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:37.499019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e55e8e-17fd-496c-bdc5-55e6cd3e5e4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:37.499757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e55e8e-17fd-496c-bdc5-55e6cd3e5e4b None admin] RBAC: Adding query filter params (role.id=45b67ba9922c4b16baae3930d3de1b6c, scope.domain.id=default, user.id=92acc9f005ce4ccfaadbc688dbb479a3) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:37.500097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e55e8e-17fd-496c-bdc5-55e6cd3e5e4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:37.501565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e55e8e-17fd-496c-bdc5-55e6cd3e5e4b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:37.511636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 22/42] 10.222.0.27 () {56 vars in 1322 bytes} [Fri Oct 14 17:41:37 2022] GET /identity/v3/role_assignments?user.id=92acc9f005ce4ccfaadbc688dbb479a3&scope.domain.id=default&role.id=45b67ba9922c4b16baae3930d3de1b6c => generated 530 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:37.561312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 21/43] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:37.701815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e55e8e-17fd-496c-bdc5-55e6cd3e5e4b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:37.785218 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:37.811336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e55e8e-17fd-496c-bdc5-55e6cd3e5e4b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:37.833841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88059f05-182b-4c43-8a9c-3e9481579c6e None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:37.843712 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2877d-0f02-44da-8f73-737913920e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:37.844345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2877d-0f02-44da-8f73-737913920e5a None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:37.844622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2877d-0f02-44da-8f73-737913920e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:37.844898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2877d-0f02-44da-8f73-737913920e5a None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:37.845565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d2877d-0f02-44da-8f73-737913920e5a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:37.846958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d2877d-0f02-44da-8f73-737913920e5a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:37.862385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 23/44] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Oct 14 17:41:37 2022] POST /identity/v3/roles => generated 223 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:41:37.941344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88059f05-182b-4c43-8a9c-3e9481579c6e None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:37.972288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:37.978287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d184e-0b91-4b82-a333-cc8cdcb32f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:37.978936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d184e-0b91-4b82-a333-cc8cdcb32f13 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:37.979213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d184e-0b91-4b82-a333-cc8cdcb32f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:37.979484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d184e-0b91-4b82-a333-cc8cdcb32f13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:37.980668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d184e-0b91-4b82-a333-cc8cdcb32f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:37.982147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d184e-0b91-4b82-a333-cc8cdcb32f13 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:37.983772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 22/45] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:41:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:41:37.989537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2877d-0f02-44da-8f73-737913920e5a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:38.096268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2877d-0f02-44da-8f73-737913920e5a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:38.128958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e1c83-ebb9-4e07-bca4-d522a1a7a648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:38.129570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e1c83-ebb9-4e07-bca4-d522a1a7a648 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:38.129848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e1c83-ebb9-4e07-bca4-d522a1a7a648 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:38.130118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e1c83-ebb9-4e07-bca4-d522a1a7a648 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:38.130755 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e1c83-ebb9-4e07-bca4-d522a1a7a648 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:38.132892 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e1c83-ebb9-4e07-bca4-d522a1a7a648 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:38.137893 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:38.161178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 24/46] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:41:37 2022] POST /identity/v3/projects => generated 312 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:41:40.321464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d527a28-fae1-463a-9d2b-ec8b9b341374 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:40.322015 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d527a28-fae1-463a-9d2b-ec8b9b341374 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:40.322015 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d527a28-fae1-463a-9d2b-ec8b9b341374 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:40.323345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 23/47] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:40.328543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8d36-4733-401e-a1e9-661de98b60dd None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:40.328938 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8d36-4733-401e-a1e9-661de98b60dd None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:40.329098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8d36-4733-401e-a1e9-661de98b60dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:40.347697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-0cff8d36-4733-401e-a1e9-661de98b60dd None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:40.349236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0cff8d36-4733-401e-a1e9-661de98b60dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:40.414295 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 25/48] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:41:40.681514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d527a28-fae1-463a-9d2b-ec8b9b341374 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:40.780117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d527a28-fae1-463a-9d2b-ec8b9b341374 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:40.813249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b98465-7679-4086-ad59-c3dd5fdff1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:40.813859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b98465-7679-4086-ad59-c3dd5fdff1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:40.814136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b98465-7679-4086-ad59-c3dd5fdff1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:40.814407 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b98465-7679-4086-ad59-c3dd5fdff1d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:40.815503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b98465-7679-4086-ad59-c3dd5fdff1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:40.816971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b98465-7679-4086-ad59-c3dd5fdff1d1 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:40.818505 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 24/49] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:41:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:40.824071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff8d36-4733-401e-a1e9-661de98b60dd None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:40.920761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff8d36-4733-401e-a1e9-661de98b60dd None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:40.937123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:40.955229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f8ef7-eabe-4068-b2ea-e0aada15bff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:40.955827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f8ef7-eabe-4068-b2ea-e0aada15bff5 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:40.956131 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f8ef7-eabe-4068-b2ea-e0aada15bff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:40.956403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f8ef7-eabe-4068-b2ea-e0aada15bff5 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:40.957046 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f8ef7-eabe-4068-b2ea-e0aada15bff5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:40.958475 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f8ef7-eabe-4068-b2ea-e0aada15bff5 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:40.983960 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 26/50] 10.222.0.27 () {60 vars in 1140 bytes} [Fri Oct 14 17:41:40 2022] POST /identity/v3/projects => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:41:43.161732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955a3ff-51bf-4c88-9723-188277a0371b None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:43.162249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955a3ff-51bf-4c88-9723-188277a0371b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:43.162249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955a3ff-51bf-4c88-9723-188277a0371b None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:43.164151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 25/51] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:41:43.169255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec0dd-9ed5-4003-8e34-8c83771e2cd8 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:43.169624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec0dd-9ed5-4003-8e34-8c83771e2cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:43.169803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec0dd-9ed5-4003-8e34-8c83771e2cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:43.188150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a84ec0dd-9ed5-4003-8e34-8c83771e2cd8 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:43.189323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a84ec0dd-9ed5-4003-8e34-8c83771e2cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:43.357644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 27/52] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:43.642260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955a3ff-51bf-4c88-9723-188277a0371b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:43.708077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:43.742046 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955a3ff-51bf-4c88-9723-188277a0371b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:43.775390 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5de6-89ea-4afc-9cb5-1d4e2ab3adae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:43.776033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5de6-89ea-4afc-9cb5-1d4e2ab3adae None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:43.776312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5de6-89ea-4afc-9cb5-1d4e2ab3adae None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:43.776580 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5de6-89ea-4afc-9cb5-1d4e2ab3adae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:43.777676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5de6-89ea-4afc-9cb5-1d4e2ab3adae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:43.779126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5de6-89ea-4afc-9cb5-1d4e2ab3adae None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:43.780694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 26/53] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:41:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:43.786276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ec0dd-9ed5-4003-8e34-8c83771e2cd8 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:43.884981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:43.889321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ec0dd-9ed5-4003-8e34-8c83771e2cd8 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:43.922774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d061ad4-03a7-4f2c-b5d6-37b1d3fddbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:43.923312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d061ad4-03a7-4f2c-b5d6-37b1d3fddbc2 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:43.923586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d061ad4-03a7-4f2c-b5d6-37b1d3fddbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:43.923856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d061ad4-03a7-4f2c-b5d6-37b1d3fddbc2 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:43.924533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d061ad4-03a7-4f2c-b5d6-37b1d3fddbc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:43.925954 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d061ad4-03a7-4f2c-b5d6-37b1d3fddbc2 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:43.951343 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 28/54] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:41:43 2022] POST /identity/v3/users => generated 275 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:41:47.268684 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca4bd0-29ad-4202-ae5d-8aed66a92d35 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:47.269738 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca4bd0-29ad-4202-ae5d-8aed66a92d35 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:47.270215 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca4bd0-29ad-4202-ae5d-8aed66a92d35 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:47.271751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 27/55] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:47.278342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a806d-b5cc-4360-80dd-3e11fd6fd4ac None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:47.279150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a806d-b5cc-4360-80dd-3e11fd6fd4ac None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:47.279642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a806d-b5cc-4360-80dd-3e11fd6fd4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:47.304881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ba1a806d-b5cc-4360-80dd-3e11fd6fd4ac None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:47.306459 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ba1a806d-b5cc-4360-80dd-3e11fd6fd4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:47.336490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a629b-eb51-4341-ae95-1d6d1f10ddf5 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:47.337090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a629b-eb51-4341-ae95-1d6d1f10ddf5 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:47.337647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a629b-eb51-4341-ae95-1d6d1f10ddf5 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:47.339172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 28/56] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:41:47.349725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c7f47-db77-4066-88aa-d91ddd4b3ec5 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:47.350817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c7f47-db77-4066-88aa-d91ddd4b3ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:47.351298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c7f47-db77-4066-88aa-d91ddd4b3ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:47.380043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e03c7f47-db77-4066-88aa-d91ddd4b3ec5 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:47.382229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e03c7f47-db77-4066-88aa-d91ddd4b3ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:47.428619 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 29/57] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:47.493204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 29/58] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:41:47.553565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b79a0-8ae8-4063-93d5-309c484ef4c5 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:47.553820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b79a0-8ae8-4063-93d5-309c484ef4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:47.554089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b79a0-8ae8-4063-93d5-309c484ef4c5 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:47.555448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 30/59] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:47.573991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c21a7-8d3d-47ab-a143-f6a24327b797 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:47.574276 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c21a7-8d3d-47ab-a143-f6a24327b797 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:47.574551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c21a7-8d3d-47ab-a143-f6a24327b797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:47.601396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-234c21a7-8d3d-47ab-a143-f6a24327b797 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:47.602615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-234c21a7-8d3d-47ab-a143-f6a24327b797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:47.690559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:47.705283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 30/60] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:47.764459 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc6d15-68d5-4b8c-98f1-75c9fc80fe25 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:47.765061 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc6d15-68d5-4b8c-98f1-75c9fc80fe25 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:47.765532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc6d15-68d5-4b8c-98f1-75c9fc80fe25 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:47.767047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 31/61] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:47.772785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfc475-4a48-4622-af2f-aa4caa5527b4 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:47.773827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc6d15-68d5-4b8c-98f1-75c9fc80fe25 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:47.777833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfc475-4a48-4622-af2f-aa4caa5527b4 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:47.778383 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfc475-4a48-4622-af2f-aa4caa5527b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:47.816775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4ecfc475-4a48-4622-af2f-aa4caa5527b4 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:47.817932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4ecfc475-4a48-4622-af2f-aa4caa5527b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:47.902641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:47.907925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 31/62] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:47.907925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfc475-4a48-4622-af2f-aa4caa5527b4 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:47.949286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc6d15-68d5-4b8c-98f1-75c9fc80fe25 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.030264 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfc475-4a48-4622-af2f-aa4caa5527b4 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.034659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf66d3-77f4-4a90-a7aa-dc3dcd9afd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.035299 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf66d3-77f4-4a90-a7aa-dc3dcd9afd93 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.035572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf66d3-77f4-4a90-a7aa-dc3dcd9afd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.035841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf66d3-77f4-4a90-a7aa-dc3dcd9afd93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.051602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf66d3-77f4-4a90-a7aa-dc3dcd9afd93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-f5cf66d3-77f4-4a90-a7aa-dc3dcd9afd93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:48.058399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:48.059248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:41:48.060034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 32/63] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:48.062401 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf66d3-77f4-4a90-a7aa-dc3dcd9afd93 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.065961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:48.076295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25fc2-c26f-4de4-8380-92e0c58cf872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.077213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25fc2-c26f-4de4-8380-92e0c58cf872 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.077813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25fc2-c26f-4de4-8380-92e0c58cf872 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.078144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25fc2-c26f-4de4-8380-92e0c58cf872 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.090354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d25fc2-c26f-4de4-8380-92e0c58cf872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-87d25fc2-c26f-4de4-8380-92e0c58cf872 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:48.095633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:48.096484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:41:48.097242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:48.099654 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 32/64] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:41:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:48.101328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25fc2-c26f-4de4-8380-92e0c58cf872 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.185198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf66d3-77f4-4a90-a7aa-dc3dcd9afd93 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.205715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25fc2-c26f-4de4-8380-92e0c58cf872 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.221158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a803e2-8ae0-4e84-a8d2-dcecfeddfa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.221791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a803e2-8ae0-4e84-a8d2-dcecfeddfa0b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.222067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a803e2-8ae0-4e84-a8d2-dcecfeddfa0b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.222339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a803e2-8ae0-4e84-a8d2-dcecfeddfa0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.233111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a803e2-8ae0-4e84-a8d2-dcecfeddfa0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-08a803e2-8ae0-4e84-a8d2-dcecfeddfa0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:48.237925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:48.238767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:41:48.239514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 33/65] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:48.240195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a803e2-8ae0-4e84-a8d2-dcecfeddfa0b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.244058 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:48.244616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c0be2-6a9c-46d1-93e9-3581dbbd4775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.245332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c0be2-6a9c-46d1-93e9-3581dbbd4775 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.245648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c0be2-6a9c-46d1-93e9-3581dbbd4775 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.245955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c0be2-6a9c-46d1-93e9-3581dbbd4775 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.246658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c0be2-6a9c-46d1-93e9-3581dbbd4775 None admin] RBAC: Adding query filter params (name=member) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:48.246980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c0be2-6a9c-46d1-93e9-3581dbbd4775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:48.248414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c0be2-6a9c-46d1-93e9-3581dbbd4775 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:48.257797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 33/66] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:48.259055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f555feb-f529-4fe5-bde7-477452595476 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.259342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f555feb-f529-4fe5-bde7-477452595476 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.259614 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f555feb-f529-4fe5-bde7-477452595476 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.261117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 34/67] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:48.261806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f555feb-f529-4fe5-bde7-477452595476 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.311530 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:48.349870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a803e2-8ae0-4e84-a8d2-dcecfeddfa0b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.363803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f555feb-f529-4fe5-bde7-477452595476 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.384495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f9a90-c1bb-4512-ae5b-3b7ef3eea7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.385099 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f9a90-c1bb-4512-ae5b-3b7ef3eea7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.385374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f9a90-c1bb-4512-ae5b-3b7ef3eea7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.385652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f9a90-c1bb-4512-ae5b-3b7ef3eea7a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.387978 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f9a90-c1bb-4512-ae5b-3b7ef3eea7a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:48.387978 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f9a90-c1bb-4512-ae5b-3b7ef3eea7a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:48.389178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f9a90-c1bb-4512-ae5b-3b7ef3eea7a7 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:48.398930 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 34/68] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:48.399983 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f9a90-c1bb-4512-ae5b-3b7ef3eea7a7 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.404040 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0803e-2c8a-4831-9f08-27539e200ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.404666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0803e-2c8a-4831-9f08-27539e200ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.404943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0803e-2c8a-4831-9f08-27539e200ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.405211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0803e-2c8a-4831-9f08-27539e200ac1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.415794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0803e-2c8a-4831-9f08-27539e200ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-7ac0803e-2c8a-4831-9f08-27539e200ac1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:48.420996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:48.421876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:48.422632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 35/69] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:48.423245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0803e-2c8a-4831-9f08-27539e200ac1 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.468953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:48.503297 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f9a90-c1bb-4512-ae5b-3b7ef3eea7a7 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.521829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0803e-2c8a-4831-9f08-27539e200ac1 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.531281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:48.539141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd5dcd-ca81-49ed-953b-6e8f295999d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.539746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd5dcd-ca81-49ed-953b-6e8f295999d0 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.540053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd5dcd-ca81-49ed-953b-6e8f295999d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.540320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd5dcd-ca81-49ed-953b-6e8f295999d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.540973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd5dcd-ca81-49ed-953b-6e8f295999d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:48.541271 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd5dcd-ca81-49ed-953b-6e8f295999d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:48.542635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd5dcd-ca81-49ed-953b-6e8f295999d0 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:48.550223 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 35/70] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:48.551342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09d75d-d31e-42cf-844b-1172d472679d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.551633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09d75d-d31e-42cf-844b-1172d472679d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.551916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09d75d-d31e-42cf-844b-1172d472679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.556494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cc81c-8b4b-4888-b139-351541d5992a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.557118 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cc81c-8b4b-4888-b139-351541d5992a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.557394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cc81c-8b4b-4888-b139-351541d5992a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.557686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cc81c-8b4b-4888-b139-351541d5992a None admin] PATH_INFO: `/v3/users/cab8f0baebc8466da5e21e19f829447a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.570926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-8d09d75d-d31e-42cf-844b-1172d472679d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:48.571694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cc81c-8b4b-4888-b139-351541d5992a None admin] RBAC: Authorizing `identity:get_user(user_id=cab8f0baebc8466da5e21e19f829447a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:48.572142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8d09d75d-d31e-42cf-844b-1172d472679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:48.573194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cc81c-8b4b-4888-b139-351541d5992a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:48.575299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 36/71] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/users/cab8f0baebc8466da5e21e19f829447a => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:48.575627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cc81c-8b4b-4888-b139-351541d5992a None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.638004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 36/72] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:48.638756 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09d75d-d31e-42cf-844b-1172d472679d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.693181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cc81c-8b4b-4888-b139-351541d5992a None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.697232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:48.731011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b17f52-4bc1-481b-a0dc-926f65c999bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.732197 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b17f52-4bc1-481b-a0dc-926f65c999bb None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.732759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b17f52-4bc1-481b-a0dc-926f65c999bb None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.733174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b17f52-4bc1-481b-a0dc-926f65c999bb None admin] PATH_INFO: `/v3/users/92acc9f005ce4ccfaadbc688dbb479a3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.734423 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b17f52-4bc1-481b-a0dc-926f65c999bb None admin] RBAC: Authorizing `identity:get_user(user_id=92acc9f005ce4ccfaadbc688dbb479a3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:48.736059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b17f52-4bc1-481b-a0dc-926f65c999bb None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:48.737756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 37/73] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/users/92acc9f005ce4ccfaadbc688dbb479a3 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:48.738641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b17f52-4bc1-481b-a0dc-926f65c999bb None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.740129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09d75d-d31e-42cf-844b-1172d472679d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.776309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:48.781362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869aa11-ecb0-4b79-a5b0-00cde86f0c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.781975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869aa11-ecb0-4b79-a5b0-00cde86f0c8e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.782254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869aa11-ecb0-4b79-a5b0-00cde86f0c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.782526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869aa11-ecb0-4b79-a5b0-00cde86f0c8e None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.783190 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869aa11-ecb0-4b79-a5b0-00cde86f0c8e None admin] RBAC: Adding query filter params (name=member) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:48.783494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869aa11-ecb0-4b79-a5b0-00cde86f0c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:48.784896 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869aa11-ecb0-4b79-a5b0-00cde86f0c8e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:48.793564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 37/74] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:48.794362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869aa11-ecb0-4b79-a5b0-00cde86f0c8e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.856487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b17f52-4bc1-481b-a0dc-926f65c999bb None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.894049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa916fb-c5c9-474e-81be-65f46a96ee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.894606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa916fb-c5c9-474e-81be-65f46a96ee1a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.894888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa916fb-c5c9-474e-81be-65f46a96ee1a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.895364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa916fb-c5c9-474e-81be-65f46a96ee1a None admin] PATH_INFO: `/v3/users/cab8f0baebc8466da5e21e19f829447a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.896332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa916fb-c5c9-474e-81be-65f46a96ee1a None admin] RBAC: Authorizing `identity:get_user(user_id=cab8f0baebc8466da5e21e19f829447a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:48.897840 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa916fb-c5c9-474e-81be-65f46a96ee1a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:48.899420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 38/75] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/users/cab8f0baebc8466da5e21e19f829447a => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:48.904026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa916fb-c5c9-474e-81be-65f46a96ee1a None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:48.904144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869aa11-ecb0-4b79-a5b0-00cde86f0c8e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:48.923441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:48.938152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f27f6-77d7-4fe6-91c3-694bbd9d3a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:48.938781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f27f6-77d7-4fe6-91c3-694bbd9d3a9a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:48.939058 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f27f6-77d7-4fe6-91c3-694bbd9d3a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:48.939329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f27f6-77d7-4fe6-91c3-694bbd9d3a9a None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:48.940477 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f27f6-77d7-4fe6-91c3-694bbd9d3a9a None admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:48.941928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f27f6-77d7-4fe6-91c3-694bbd9d3a9a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:48.943783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 38/76] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:48.944975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f27f6-77d7-4fe6-91c3-694bbd9d3a9a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:49.005297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa916fb-c5c9-474e-81be-65f46a96ee1a None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:49.006861 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:49.040416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3d690-acf3-4f07-9814-01510defb579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:49.041038 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d690-acf3-4f07-9814-01510defb579 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:49.041323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d690-acf3-4f07-9814-01510defb579 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:49.041591 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d690-acf3-4f07-9814-01510defb579 None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:49.042707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3d690-acf3-4f07-9814-01510defb579 None admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:49.044206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3d690-acf3-4f07-9814-01510defb579 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:49.045833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 39/77] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:49.046589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3d690-acf3-4f07-9814-01510defb579 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:49.046839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f27f6-77d7-4fe6-91c3-694bbd9d3a9a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:49.085813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d973d-f3ee-4b27-9ef4-46db37fba023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:49.086441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d973d-f3ee-4b27-9ef4-46db37fba023 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:49.086723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d973d-f3ee-4b27-9ef4-46db37fba023 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:49.087021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d973d-f3ee-4b27-9ef4-46db37fba023 None admin] PATH_INFO: `/v3/users/cab8f0baebc8466da5e21e19f829447a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:49.088187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d973d-f3ee-4b27-9ef4-46db37fba023 None admin] RBAC: Authorizing `identity:get_user(user_id=cab8f0baebc8466da5e21e19f829447a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:49.089639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d973d-f3ee-4b27-9ef4-46db37fba023 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:49.091205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 39/78] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:48 2022] GET /identity/v3/users/cab8f0baebc8466da5e21e19f829447a => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:49.096445 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d973d-f3ee-4b27-9ef4-46db37fba023 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:49.146045 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3d690-acf3-4f07-9814-01510defb579 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:49.152492 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:49.182025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426ca36-0663-4306-9d34-f7314d83f536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:49.182650 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426ca36-0663-4306-9d34-f7314d83f536 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:49.182930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426ca36-0663-4306-9d34-f7314d83f536 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:49.183199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426ca36-0663-4306-9d34-f7314d83f536 None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:49.184339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426ca36-0663-4306-9d34-f7314d83f536 None admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:49.185781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426ca36-0663-4306-9d34-f7314d83f536 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:49.187408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 40/79] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:41:49 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:49.187408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426ca36-0663-4306-9d34-f7314d83f536 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:49.199303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d973d-f3ee-4b27-9ef4-46db37fba023 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:49.228110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:49.233014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619958e-0fd9-4fd7-a0b2-4f53eea009c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:49.233634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619958e-0fd9-4fd7-a0b2-4f53eea009c1 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:49.233909 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619958e-0fd9-4fd7-a0b2-4f53eea009c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:49.234178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619958e-0fd9-4fd7-a0b2-4f53eea009c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:49.235287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619958e-0fd9-4fd7-a0b2-4f53eea009c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:49.236748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619958e-0fd9-4fd7-a0b2-4f53eea009c1 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:49.238289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 40/80] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:41:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:49.239011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619958e-0fd9-4fd7-a0b2-4f53eea009c1 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:49.289907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426ca36-0663-4306-9d34-f7314d83f536 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:49.323243 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3089d-b63b-40b7-9cc5-eaa8a6e9abf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:49.323947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3089d-b63b-40b7-9cc5-eaa8a6e9abf5 None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:49.324189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3089d-b63b-40b7-9cc5-eaa8a6e9abf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:49.324456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3089d-b63b-40b7-9cc5-eaa8a6e9abf5 None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/users/cab8f0baebc8466da5e21e19f829447a/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:49.326322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3089d-b63b-40b7-9cc5-eaa8a6e9abf5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e0511b004c94f1bbed48dd4fd34cb6a, user_id=cab8f0baebc8466da5e21e19f829447a, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:49.327747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3089d-b63b-40b7-9cc5-eaa8a6e9abf5 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:49.338501 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619958e-0fd9-4fd7-a0b2-4f53eea009c1 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:49.340768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 41/81] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:41:49 2022] PUT /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/users/cab8f0baebc8466da5e21e19f829447a/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:41:49.341552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3089d-b63b-40b7-9cc5-eaa8a6e9abf5 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:49.373970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:49.390056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a3a8-2195-42f8-a726-d111f78633ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:49.391070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a3a8-2195-42f8-a726-d111f78633ce None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:49.391351 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a3a8-2195-42f8-a726-d111f78633ce None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:49.391633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a3a8-2195-42f8-a726-d111f78633ce None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/users/92acc9f005ce4ccfaadbc688dbb479a3/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:49.393855 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a3a8-2195-42f8-a726-d111f78633ce None admin] RBAC: Authorizing `identity:create_grant(project_id=6e0511b004c94f1bbed48dd4fd34cb6a, user_id=92acc9f005ce4ccfaadbc688dbb479a3, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:49.395828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a3a8-2195-42f8-a726-d111f78633ce None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:49.422169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 41/82] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:41:49 2022] PUT /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/users/92acc9f005ce4ccfaadbc688dbb479a3/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:41:49.422169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a3a8-2195-42f8-a726-d111f78633ce None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:49.484925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:49.489083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3089d-b63b-40b7-9cc5-eaa8a6e9abf5 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:49.540163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ad1d3-a9eb-45c5-b2f8-0800ba117d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:49.540887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ad1d3-a9eb-45c5-b2f8-0800ba117d72 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:49.541176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ad1d3-a9eb-45c5-b2f8-0800ba117d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:49.541487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ad1d3-a9eb-45c5-b2f8-0800ba117d72 None admin] PATH_INFO: `/v3/projects/4200f99fc3a741e490b1eb13e1004267` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:49.542691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ad1d3-a9eb-45c5-b2f8-0800ba117d72 None admin] RBAC: Authorizing `identity:get_project(project_id=4200f99fc3a741e490b1eb13e1004267)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:49.544211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ad1d3-a9eb-45c5-b2f8-0800ba117d72 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:49.546076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 42/83] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:41:49 2022] GET /identity/v3/projects/4200f99fc3a741e490b1eb13e1004267 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:49.546701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ad1d3-a9eb-45c5-b2f8-0800ba117d72 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:49.555354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a3a8-2195-42f8-a726-d111f78633ce None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:49.606607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394a7fe-083a-45b9-800a-e58bd9b96e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:49.607289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394a7fe-083a-45b9-800a-e58bd9b96e8a None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:49.607569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394a7fe-083a-45b9-800a-e58bd9b96e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:49.607941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394a7fe-083a-45b9-800a-e58bd9b96e8a None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/users/cab8f0baebc8466da5e21e19f829447a/roles/1cc2ea78f84c4491967839cfead3fd26` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:49.610897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394a7fe-083a-45b9-800a-e58bd9b96e8a None admin] RBAC: Authorizing `identity:create_grant(project_id=6e0511b004c94f1bbed48dd4fd34cb6a, user_id=cab8f0baebc8466da5e21e19f829447a, role_id=1cc2ea78f84c4491967839cfead3fd26)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:49.612697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394a7fe-083a-45b9-800a-e58bd9b96e8a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:49.627509 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 42/84] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:41:49 2022] PUT /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/users/cab8f0baebc8466da5e21e19f829447a/roles/1cc2ea78f84c4491967839cfead3fd26 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:41:49.628335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394a7fe-083a-45b9-800a-e58bd9b96e8a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:49.688156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ad1d3-a9eb-45c5-b2f8-0800ba117d72 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:49.693732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:49.736409 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd21bf-a6b4-42e6-bd58-a5451bb322c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:49.737085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd21bf-a6b4-42e6-bd58-a5451bb322c3 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:49.737370 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd21bf-a6b4-42e6-bd58-a5451bb322c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:49.737639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd21bf-a6b4-42e6-bd58-a5451bb322c3 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:49.738679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd21bf-a6b4-42e6-bd58-a5451bb322c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:49.739951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd21bf-a6b4-42e6-bd58-a5451bb322c3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:49.764526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394a7fe-083a-45b9-800a-e58bd9b96e8a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:49.780088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 43/85] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:41:49 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:41:49.832653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0ed6e-8da4-457a-bb49-ea9c7ab68f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:49.833685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0ed6e-8da4-457a-bb49-ea9c7ab68f96 None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:49.834216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0ed6e-8da4-457a-bb49-ea9c7ab68f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:49.835270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0ed6e-8da4-457a-bb49-ea9c7ab68f96 None admin] PATH_INFO: `/v3/projects/4200f99fc3a741e490b1eb13e1004267/users/cab8f0baebc8466da5e21e19f829447a/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:49.840097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0ed6e-8da4-457a-bb49-ea9c7ab68f96 None admin] RBAC: Authorizing `identity:create_grant(project_id=4200f99fc3a741e490b1eb13e1004267, user_id=cab8f0baebc8466da5e21e19f829447a, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:49.842096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0ed6e-8da4-457a-bb49-ea9c7ab68f96 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:49.859624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 43/86] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:41:49 2022] PUT /identity/v3/projects/4200f99fc3a741e490b1eb13e1004267/users/cab8f0baebc8466da5e21e19f829447a/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:41:51.993062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd3b8d-e374-4be6-80f6-d54f0a97524a None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:51.994063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd3b8d-e374-4be6-80f6-d54f0a97524a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:51.994535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd3b8d-e374-4be6-80f6-d54f0a97524a None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:51.996376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 44/87] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:52.001369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1b54c-dab7-4a32-8e90-8b6436ea8836 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:52.002160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1b54c-dab7-4a32-8e90-8b6436ea8836 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:52.002626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1b54c-dab7-4a32-8e90-8b6436ea8836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:52.028552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a4c1b54c-dab7-4a32-8e90-8b6436ea8836 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:52.030062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a4c1b54c-dab7-4a32-8e90-8b6436ea8836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:52.125531 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 44/88] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:52.487805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de137d3-36ea-4f13-9746-8b811736ceba None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:52.488538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de137d3-36ea-4f13-9746-8b811736ceba None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:52.489025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de137d3-36ea-4f13-9746-8b811736ceba None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:52.491323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 45/89] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:52.509645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566d88-cbaf-4fae-bb6c-b55a471404a1 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:52.510255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566d88-cbaf-4fae-bb6c-b55a471404a1 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:52.510728 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566d88-cbaf-4fae-bb6c-b55a471404a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:52.540902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c3566d88-cbaf-4fae-bb6c-b55a471404a1 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:52.542451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c3566d88-cbaf-4fae-bb6c-b55a471404a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:52.571340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:52.606680 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e8698-b465-4941-8f95-a11722deb58f None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:52.607281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e8698-b465-4941-8f95-a11722deb58f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:52.607754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e8698-b465-4941-8f95-a11722deb58f None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:52.609413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 46/90] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:41:52.615059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d47590-a4fe-4ce2-a450-bd1772be18c0 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:52.616161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d47590-a4fe-4ce2-a450-bd1772be18c0 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:52.616686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d47590-a4fe-4ce2-a450-bd1772be18c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:52.645040 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-a3d47590-a4fe-4ce2-a450-bd1772be18c0 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:52.648289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a3d47590-a4fe-4ce2-a450-bd1772be18c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:52.658005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 45/91] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:52.684084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:52.686766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3566d88-cbaf-4fae-bb6c-b55a471404a1 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:52.756619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 47/92] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:52.777979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058419ab-d48f-40cf-bbdf-467a8d83787e None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:52.778613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058419ab-d48f-40cf-bbdf-467a8d83787e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:52.779078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058419ab-d48f-40cf-bbdf-467a8d83787e None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:52.781202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 48/93] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:41:52.790867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66696c2-8730-46f3-8c66-c1746dac10df None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:52.791388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66696c2-8730-46f3-8c66-c1746dac10df None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:52.791854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66696c2-8730-46f3-8c66-c1746dac10df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:52.828678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f66696c2-8730-46f3-8c66-c1746dac10df None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:52.830187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f66696c2-8730-46f3-8c66-c1746dac10df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:52.884293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3566d88-cbaf-4fae-bb6c-b55a471404a1 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:52.937905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952ae02-e4b3-4f48-bdcc-62b3fde54f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:52.938535 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952ae02-e4b3-4f48-bdcc-62b3fde54f5e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:52.938811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952ae02-e4b3-4f48-bdcc-62b3fde54f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:52.939079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952ae02-e4b3-4f48-bdcc-62b3fde54f5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:52.954108 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 49/94] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:41:52.957866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952ae02-e4b3-4f48-bdcc-62b3fde54f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-e952ae02-e4b3-4f48-bdcc-62b3fde54f5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:52.963229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:52.964083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:41:52.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 46/95] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:41:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:52.971909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66696c2-8730-46f3-8c66-c1746dac10df None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:53.031992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952ae02-e4b3-4f48-bdcc-62b3fde54f5e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:53.061191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:53.108404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:53.146710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66696c2-8730-46f3-8c66-c1746dac10df None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:53.191136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952ae02-e4b3-4f48-bdcc-62b3fde54f5e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:53.224607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9276b86-5de3-4fa4-81cc-cc640d3b5538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:53.225350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9276b86-5de3-4fa4-81cc-cc640d3b5538 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:53.225634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9276b86-5de3-4fa4-81cc-cc640d3b5538 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:53.225909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9276b86-5de3-4fa4-81cc-cc640d3b5538 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:53.226592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9276b86-5de3-4fa4-81cc-cc640d3b5538 None admin] RBAC: Adding query filter params (name=member) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:53.226892 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9276b86-5de3-4fa4-81cc-cc640d3b5538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:53.228357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9276b86-5de3-4fa4-81cc-cc640d3b5538 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:53.229146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed5232-d862-4572-9de8-c096d9939361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:53.229777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5232-d862-4572-9de8-c096d9939361 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:53.230052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5232-d862-4572-9de8-c096d9939361 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:53.230325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5232-d862-4572-9de8-c096d9939361 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:53.241807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed5232-d862-4572-9de8-c096d9939361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:53.244250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 50/96] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:41:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:53.245111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9276b86-5de3-4fa4-81cc-cc640d3b5538 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-6fed5232-d862-4572-9de8-c096d9939361 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:53.246665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:53.247711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:41:53.248753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 47/97] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:41:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:53.253167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed5232-d862-4572-9de8-c096d9939361 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:53.351906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:53.366158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed5232-d862-4572-9de8-c096d9939361 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:53.402844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9276b86-5de3-4fa4-81cc-cc640d3b5538 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:53.407136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c24f-5588-478a-9099-afffbc95848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:53.407770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c24f-5588-478a-9099-afffbc95848d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:53.408090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c24f-5588-478a-9099-afffbc95848d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:53.408363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c24f-5588-478a-9099-afffbc95848d None admin] PATH_INFO: `/v3/users/cab8f0baebc8466da5e21e19f829447a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:53.409511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38c24f-5588-478a-9099-afffbc95848d None admin] RBAC: Authorizing `identity:get_user(user_id=cab8f0baebc8466da5e21e19f829447a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:53.411101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38c24f-5588-478a-9099-afffbc95848d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:53.414975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 48/98] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:53 2022] GET /identity/v3/users/cab8f0baebc8466da5e21e19f829447a => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:53.415783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c24f-5588-478a-9099-afffbc95848d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:53.440462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3466a-ab62-4a6e-9666-59148dbf1190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:53.441150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3466a-ab62-4a6e-9666-59148dbf1190 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:53.441437 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3466a-ab62-4a6e-9666-59148dbf1190 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:53.441712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3466a-ab62-4a6e-9666-59148dbf1190 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:53.453760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3466a-ab62-4a6e-9666-59148dbf1190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-5fd3466a-ab62-4a6e-9666-59148dbf1190 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:53.459870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:53.460793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 51/99] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:41:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:53.462287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3466a-ab62-4a6e-9666-59148dbf1190 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:53.467465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:53.539862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c24f-5588-478a-9099-afffbc95848d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:53.588035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4127f4-516e-4d9c-80de-250679769af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:53.588661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4127f4-516e-4d9c-80de-250679769af6 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:53.588940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4127f4-516e-4d9c-80de-250679769af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:53.589219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4127f4-516e-4d9c-80de-250679769af6 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:53.589438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3466a-ab62-4a6e-9666-59148dbf1190 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:53.589894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4127f4-516e-4d9c-80de-250679769af6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:53.590293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4127f4-516e-4d9c-80de-250679769af6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:53.592992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4127f4-516e-4d9c-80de-250679769af6 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:53.604493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 49/100] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:41:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:53.605844 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4127f4-516e-4d9c-80de-250679769af6 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:53.627091 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:53.633121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c637dc-08a0-418f-a249-177d77819e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:53.633924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c637dc-08a0-418f-a249-177d77819e30 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:53.634313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c637dc-08a0-418f-a249-177d77819e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:53.634710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c637dc-08a0-418f-a249-177d77819e30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:53.645931 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c637dc-08a0-418f-a249-177d77819e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-35c637dc-08a0-418f-a249-177d77819e30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:53.652265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:53.654099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 52/101] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:41:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:53.656154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c637dc-08a0-418f-a249-177d77819e30 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:53.736252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4127f4-516e-4d9c-80de-250679769af6 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:53.761370 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:53.781897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06907fdf-82ba-4bb7-86ee-99da1bf0794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:53.782698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06907fdf-82ba-4bb7-86ee-99da1bf0794d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:53.783070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06907fdf-82ba-4bb7-86ee-99da1bf0794d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:53.783436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06907fdf-82ba-4bb7-86ee-99da1bf0794d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:53.813711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c637dc-08a0-418f-a249-177d77819e30 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:53.850611 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3bc3d-28eb-44fb-88db-b59cce3d35e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:53.851235 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3bc3d-28eb-44fb-88db-b59cce3d35e1 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:53.851516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3bc3d-28eb-44fb-88db-b59cce3d35e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:53.851792 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3bc3d-28eb-44fb-88db-b59cce3d35e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:53.852497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3bc3d-28eb-44fb-88db-b59cce3d35e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:53.852827 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3bc3d-28eb-44fb-88db-b59cce3d35e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:53.854237 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3bc3d-28eb-44fb-88db-b59cce3d35e1 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:53.857828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06907fdf-82ba-4bb7-86ee-99da1bf0794d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:53.859365 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06907fdf-82ba-4bb7-86ee-99da1bf0794d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:53.869114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 53/102] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:41:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:53.869114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3bc3d-28eb-44fb-88db-b59cce3d35e1 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:53.877219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:53.964224 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 50/103] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:41:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:41:53.965059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06907fdf-82ba-4bb7-86ee-99da1bf0794d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:53.970444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3bc3d-28eb-44fb-88db-b59cce3d35e1 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:53.995636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:54.004326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfad7a9-5d68-4615-a57f-d0786cdbc752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:54.004945 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfad7a9-5d68-4615-a57f-d0786cdbc752 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.005227 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfad7a9-5d68-4615-a57f-d0786cdbc752 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.005494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfad7a9-5d68-4615-a57f-d0786cdbc752 None admin] PATH_INFO: `/v3/users/92acc9f005ce4ccfaadbc688dbb479a3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.007172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfad7a9-5d68-4615-a57f-d0786cdbc752 None admin] RBAC: Authorizing `identity:get_user(user_id=92acc9f005ce4ccfaadbc688dbb479a3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:54.008752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfad7a9-5d68-4615-a57f-d0786cdbc752 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:54.010473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 54/104] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:53 2022] GET /identity/v3/users/92acc9f005ce4ccfaadbc688dbb479a3 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:54.011812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82053ec0-15ad-4784-882f-9fdcbe85f74d None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.012116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82053ec0-15ad-4784-882f-9fdcbe85f74d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.012386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82053ec0-15ad-4784-882f-9fdcbe85f74d None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.013557 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 55/105] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:54.016633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82053ec0-15ad-4784-882f-9fdcbe85f74d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:54.071666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06907fdf-82ba-4bb7-86ee-99da1bf0794d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:54.085457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:54.105523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f6a43-dc53-48c3-adf8-aca776dd7c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:54.106120 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f6a43-dc53-48c3-adf8-aca776dd7c1c None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.106397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f6a43-dc53-48c3-adf8-aca776dd7c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.106671 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f6a43-dc53-48c3-adf8-aca776dd7c1c None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.107332 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f6a43-dc53-48c3-adf8-aca776dd7c1c None admin] RBAC: Adding query filter params (name=member) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:54.107633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f6a43-dc53-48c3-adf8-aca776dd7c1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:54.109072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f6a43-dc53-48c3-adf8-aca776dd7c1c None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:54.117443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82053ec0-15ad-4784-882f-9fdcbe85f74d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:54.117904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 51/106] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:41:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:54.118808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b3b6e-f29e-45ed-9b02-ccd9a685c154 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.119094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b3b6e-f29e-45ed-9b02-ccd9a685c154 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.119367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b3b6e-f29e-45ed-9b02-ccd9a685c154 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.120702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 52/107] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:54.121328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b3b6e-f29e-45ed-9b02-ccd9a685c154 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:54.158692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4c976-02b6-4848-a4b5-b09de63a9197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:54.158692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4c976-02b6-4848-a4b5-b09de63a9197 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.158981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4c976-02b6-4848-a4b5-b09de63a9197 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.159201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4c976-02b6-4848-a4b5-b09de63a9197 None admin] PATH_INFO: `/v3/users/cab8f0baebc8466da5e21e19f829447a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.160366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4c976-02b6-4848-a4b5-b09de63a9197 None admin] RBAC: Authorizing `identity:get_user(user_id=cab8f0baebc8466da5e21e19f829447a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:54.161814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4c976-02b6-4848-a4b5-b09de63a9197 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:54.163446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 56/108] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:54 2022] GET /identity/v3/users/cab8f0baebc8466da5e21e19f829447a => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:54.164232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4c976-02b6-4848-a4b5-b09de63a9197 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:54.223249 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b3b6e-f29e-45ed-9b02-ccd9a685c154 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:54.228775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:54.258470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1f91d-2b76-4b5c-bb0d-d93b6e9ac7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:54.259095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1f91d-2b76-4b5c-bb0d-d93b6e9ac7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.259367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1f91d-2b76-4b5c-bb0d-d93b6e9ac7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.259637 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1f91d-2b76-4b5c-bb0d-d93b6e9ac7f7 None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.260755 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1f91d-2b76-4b5c-bb0d-d93b6e9ac7f7 None admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:54.262456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1f91d-2b76-4b5c-bb0d-d93b6e9ac7f7 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:54.264094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 53/109] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:41:54 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:54.265205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8651cb-fea5-40d8-a4f1-93d9d18ba918 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.265453 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8651cb-fea5-40d8-a4f1-93d9d18ba918 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.265724 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8651cb-fea5-40d8-a4f1-93d9d18ba918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.268269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4c976-02b6-4848-a4b5-b09de63a9197 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:54.284400 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-7f8651cb-fea5-40d8-a4f1-93d9d18ba918 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:54.285607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7f8651cb-fea5-40d8-a4f1-93d9d18ba918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:54.303016 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cc962-7377-44d6-9e04-34331daa16b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:54.303820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cc962-7377-44d6-9e04-34331daa16b7 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.304134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cc962-7377-44d6-9e04-34331daa16b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.304411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cc962-7377-44d6-9e04-34331daa16b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.326106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:54.355891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 54/110] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:54.364069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8651cb-fea5-40d8-a4f1-93d9d18ba918 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:54.381146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552cc962-7377-44d6-9e04-34331daa16b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:54.382742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552cc962-7377-44d6-9e04-34331daa16b7 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:54.455234 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:54.462531 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8651cb-fea5-40d8-a4f1-93d9d18ba918 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:54.498858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 57/111] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:41:54.500137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b63d8-a89f-46da-a46f-1ffc0198d859 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.500426 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b63d8-a89f-46da-a46f-1ffc0198d859 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.500701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b63d8-a89f-46da-a46f-1ffc0198d859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.504655 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77f111-ff9d-403b-b322-32a76d59081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:54.505292 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77f111-ff9d-403b-b322-32a76d59081d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.505569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77f111-ff9d-403b-b322-32a76d59081d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.505840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77f111-ff9d-403b-b322-32a76d59081d None admin] PATH_INFO: `/v3/users/cab8f0baebc8466da5e21e19f829447a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.507168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77f111-ff9d-403b-b322-32a76d59081d None admin] RBAC: Authorizing `identity:get_user(user_id=cab8f0baebc8466da5e21e19f829447a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:54.508657 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77f111-ff9d-403b-b322-32a76d59081d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:54.514748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 55/112] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:54 2022] GET /identity/v3/users/cab8f0baebc8466da5e21e19f829447a => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:54.514748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77f111-ff9d-403b-b322-32a76d59081d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:54.524449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8c2b63d8-a89f-46da-a46f-1ffc0198d859 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:54.525945 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8c2b63d8-a89f-46da-a46f-1ffc0198d859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:54.556214 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:54.602532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 58/113] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:54.602532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b63d8-a89f-46da-a46f-1ffc0198d859 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:54.630342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77f111-ff9d-403b-b322-32a76d59081d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:54.665213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d1b2a-27da-4cd7-a9ff-d1b16c94ad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:54.665819 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d1b2a-27da-4cd7-a9ff-d1b16c94ad8c None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.666098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d1b2a-27da-4cd7-a9ff-d1b16c94ad8c None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.666370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d1b2a-27da-4cd7-a9ff-d1b16c94ad8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.689603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:54.700328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b63d8-a89f-46da-a46f-1ffc0198d859 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:54.736401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3270c4-1d39-4eca-bd9b-9ad056164a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:54.737004 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3270c4-1d39-4eca-bd9b-9ad056164a54 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.737284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3270c4-1d39-4eca-bd9b-9ad056164a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.737558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3270c4-1d39-4eca-bd9b-9ad056164a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.738193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d1b2a-27da-4cd7-a9ff-d1b16c94ad8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:54.738292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3270c4-1d39-4eca-bd9b-9ad056164a54 None admin] RBAC: Adding query filter params (role.id=9ab8d70da5894a2f99c6d22e5351baa3, scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a, user.id=cab8f0baebc8466da5e21e19f829447a) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:54.738568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3270c4-1d39-4eca-bd9b-9ad056164a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:54.739657 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d1b2a-27da-4cd7-a9ff-d1b16c94ad8c None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:54.740036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3270c4-1d39-4eca-bd9b-9ad056164a54 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:54.747533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 59/114] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:41:54 2022] GET /identity/v3/role_assignments?user.id=cab8f0baebc8466da5e21e19f829447a&scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a&role.id=9ab8d70da5894a2f99c6d22e5351baa3 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:54.748168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3270c4-1d39-4eca-bd9b-9ad056164a54 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:54.810182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:54.890690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 56/115] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:41:54.891630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d1b2a-27da-4cd7-a9ff-d1b16c94ad8c None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:54.898580 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3270c4-1d39-4eca-bd9b-9ad056164a54 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:54.938317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63428d39-5b4d-4340-bfd5-3d7439216e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:54.938956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63428d39-5b4d-4340-bfd5-3d7439216e64 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:54.939236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63428d39-5b4d-4340-bfd5-3d7439216e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:54.939510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63428d39-5b4d-4340-bfd5-3d7439216e64 None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:54.943354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63428d39-5b4d-4340-bfd5-3d7439216e64 None admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:54.946206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63428d39-5b4d-4340-bfd5-3d7439216e64 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:54.947938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 60/116] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:41:54 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:54.948719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63428d39-5b4d-4340-bfd5-3d7439216e64 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:54.977138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:55.013149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d1b2a-27da-4cd7-a9ff-d1b16c94ad8c None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.047497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9d72f-7181-4565-9299-fd1e2060da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.048121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d72f-7181-4565-9299-fd1e2060da05 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.048424 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d72f-7181-4565-9299-fd1e2060da05 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.048693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d72f-7181-4565-9299-fd1e2060da05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.051498 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63428d39-5b4d-4340-bfd5-3d7439216e64 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.061058 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:55.085102 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fac5f-daa3-47e5-984f-7a886ee49604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.085732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fac5f-daa3-47e5-984f-7a886ee49604 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.086009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fac5f-daa3-47e5-984f-7a886ee49604 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.086281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fac5f-daa3-47e5-984f-7a886ee49604 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.096108 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fac5f-daa3-47e5-984f-7a886ee49604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-d56fac5f-daa3-47e5-984f-7a886ee49604 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:55.101914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:55.102759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:55.103516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 61/117] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:41:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:55.105110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fac5f-daa3-47e5-984f-7a886ee49604 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:55.116987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9d72f-7181-4565-9299-fd1e2060da05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:55.118461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9d72f-7181-4565-9299-fd1e2060da05 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:55.186244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:55.202802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fac5f-daa3-47e5-984f-7a886ee49604 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.215359 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 57/118] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:41:55.216180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9d72f-7181-4565-9299-fd1e2060da05 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:55.237082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4ed1d-1656-467d-882b-affe684e7d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.237703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4ed1d-1656-467d-882b-affe684e7d32 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.237982 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4ed1d-1656-467d-882b-affe684e7d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.238253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4ed1d-1656-467d-882b-affe684e7d32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.239358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4ed1d-1656-467d-882b-affe684e7d32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:55.240908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4ed1d-1656-467d-882b-affe684e7d32 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:55.242600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 62/119] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:41:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:55.243217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4ed1d-1656-467d-882b-affe684e7d32 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:55.261490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:55.316430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9d72f-7181-4565-9299-fd1e2060da05 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.348322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4ed1d-1656-467d-882b-affe684e7d32 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.358299 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16092db4-4448-4f9e-9ff7-be5f954c718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.358944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092db4-4448-4f9e-9ff7-be5f954c718c None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.359225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092db4-4448-4f9e-9ff7-be5f954c718c None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.359500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092db4-4448-4f9e-9ff7-be5f954c718c None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.360658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16092db4-4448-4f9e-9ff7-be5f954c718c None admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:55.362114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16092db4-4448-4f9e-9ff7-be5f954c718c None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:55.363837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 58/120] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:41:55 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:55.370152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16092db4-4448-4f9e-9ff7-be5f954c718c None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:55.388377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc8d69-1723-40c2-add9-90e953ace147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.391220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc8d69-1723-40c2-add9-90e953ace147 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.391513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc8d69-1723-40c2-add9-90e953ace147 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.391957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc8d69-1723-40c2-add9-90e953ace147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.392584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc8d69-1723-40c2-add9-90e953ace147 None admin] RBAC: Adding query filter params (role.id=45b67ba9922c4b16baae3930d3de1b6c, scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a, user.id=92acc9f005ce4ccfaadbc688dbb479a3) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:55.392887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc8d69-1723-40c2-add9-90e953ace147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:55.394430 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc8d69-1723-40c2-add9-90e953ace147 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:55.415956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 63/121] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:41:55 2022] GET /identity/v3/role_assignments?user.id=92acc9f005ce4ccfaadbc688dbb479a3&scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a&role.id=45b67ba9922c4b16baae3930d3de1b6c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:55.417033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc8d69-1723-40c2-add9-90e953ace147 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:55.445052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:55.496812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16092db4-4448-4f9e-9ff7-be5f954c718c None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.510138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:55.542933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390d15e-4213-420a-a73e-4baed516140a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.544031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390d15e-4213-420a-a73e-4baed516140a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.544539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390d15e-4213-420a-a73e-4baed516140a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.544716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc8d69-1723-40c2-add9-90e953ace147 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.545162 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390d15e-4213-420a-a73e-4baed516140a None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.546121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390d15e-4213-420a-a73e-4baed516140a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:55.546663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390d15e-4213-420a-a73e-4baed516140a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:55.547970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390d15e-4213-420a-a73e-4baed516140a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:55.558165 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 59/122] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:41:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:55.558827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390d15e-4213-420a-a73e-4baed516140a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:55.591472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec33ff-00e7-4ae7-84d2-d236b1214ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.592088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec33ff-00e7-4ae7-84d2-d236b1214ada None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.592365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec33ff-00e7-4ae7-84d2-d236b1214ada None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.592637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec33ff-00e7-4ae7-84d2-d236b1214ada None admin] PATH_INFO: `/v3/projects/4200f99fc3a741e490b1eb13e1004267` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.594096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec33ff-00e7-4ae7-84d2-d236b1214ada None admin] RBAC: Authorizing `identity:get_project(project_id=4200f99fc3a741e490b1eb13e1004267)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:55.595213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec33ff-00e7-4ae7-84d2-d236b1214ada None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:55.596879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 64/123] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:41:55 2022] GET /identity/v3/projects/4200f99fc3a741e490b1eb13e1004267 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:55.600104 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec33ff-00e7-4ae7-84d2-d236b1214ada None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:55.689063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390d15e-4213-420a-a73e-4baed516140a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.695860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:55.712794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec33ff-00e7-4ae7-84d2-d236b1214ada None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.723841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79867c9-edfd-4562-9a67-74ef1f854bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.724469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79867c9-edfd-4562-9a67-74ef1f854bd9 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.724751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79867c9-edfd-4562-9a67-74ef1f854bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.725553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79867c9-edfd-4562-9a67-74ef1f854bd9 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.726206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79867c9-edfd-4562-9a67-74ef1f854bd9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:55.727739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79867c9-edfd-4562-9a67-74ef1f854bd9 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:55.746940 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd386097-5350-4465-9a4f-90f6a55a3e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.747540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd386097-5350-4465-9a4f-90f6a55a3e8f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.747823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd386097-5350-4465-9a4f-90f6a55a3e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.748129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd386097-5350-4465-9a4f-90f6a55a3e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.748852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd386097-5350-4465-9a4f-90f6a55a3e8f None admin] RBAC: Adding query filter params (role.id=1cc2ea78f84c4491967839cfead3fd26, scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a, user.id=cab8f0baebc8466da5e21e19f829447a) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:55.749158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd386097-5350-4465-9a4f-90f6a55a3e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:55.750603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd386097-5350-4465-9a4f-90f6a55a3e8f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:55.753994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 60/124] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:41:55 2022] POST /identity/v3/projects => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:41:55.754812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79867c9-edfd-4562-9a67-74ef1f854bd9 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:55.768157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 65/125] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:41:55 2022] GET /identity/v3/role_assignments?user.id=cab8f0baebc8466da5e21e19f829447a&scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a&role.id=1cc2ea78f84c4491967839cfead3fd26 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:55.768157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd386097-5350-4465-9a4f-90f6a55a3e8f None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:55.796052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:55.898757 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd386097-5350-4465-9a4f-90f6a55a3e8f None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.905770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79867c9-edfd-4562-9a67-74ef1f854bd9 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:55.943419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e150b8d-9d77-42b6-9d4c-de50ef159f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.944638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e150b8d-9d77-42b6-9d4c-de50ef159f95 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.944917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e150b8d-9d77-42b6-9d4c-de50ef159f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.945193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e150b8d-9d77-42b6-9d4c-de50ef159f95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.946291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e150b8d-9d77-42b6-9d4c-de50ef159f95 None admin] RBAC: Adding query filter params (role.id=9ab8d70da5894a2f99c6d22e5351baa3, scope.project.id=4200f99fc3a741e490b1eb13e1004267, user.id=cab8f0baebc8466da5e21e19f829447a) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:55.946613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e150b8d-9d77-42b6-9d4c-de50ef159f95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:55.951993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e150b8d-9d77-42b6-9d4c-de50ef159f95 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:55.952584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b30cf6-0e3b-4e6d-aebf-7c2da54c987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:55.953240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b30cf6-0e3b-4e6d-aebf-7c2da54c987b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:55.953523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b30cf6-0e3b-4e6d-aebf-7c2da54c987b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:55.953800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b30cf6-0e3b-4e6d-aebf-7c2da54c987b None admin] PATH_INFO: `/v3/users/1aa7307e2397438abfc4aef9b8d08b51` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:55.956221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 66/126] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:41:55 2022] GET /identity/v3/role_assignments?user.id=cab8f0baebc8466da5e21e19f829447a&scope.project.id=4200f99fc3a741e490b1eb13e1004267&role.id=9ab8d70da5894a2f99c6d22e5351baa3 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:55.981059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b30cf6-0e3b-4e6d-aebf-7c2da54c987b None admin] RBAC: Authorizing `identity:get_user(user_id=1aa7307e2397438abfc4aef9b8d08b51)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:55.982572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b30cf6-0e3b-4e6d-aebf-7c2da54c987b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:55.984624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 61/127] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:55 2022] GET /identity/v3/users/1aa7307e2397438abfc4aef9b8d08b51 => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:55.990404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e150b8d-9d77-42b6-9d4c-de50ef159f95 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:56.009132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:56.110344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e150b8d-9d77-42b6-9d4c-de50ef159f95 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:56.151781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3fd52-aea0-4162-b66c-af13ce9ce63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:56.152460 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3fd52-aea0-4162-b66c-af13ce9ce63c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:56.152743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3fd52-aea0-4162-b66c-af13ce9ce63c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:56.153018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3fd52-aea0-4162-b66c-af13ce9ce63c None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:56.154222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3fd52-aea0-4162-b66c-af13ce9ce63c None admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:56.155696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3fd52-aea0-4162-b66c-af13ce9ce63c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:56.157474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 67/128] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:41:55 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:56.163931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b30cf6-0e3b-4e6d-aebf-7c2da54c987b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:56.254594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:56.273548 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b30cf6-0e3b-4e6d-aebf-7c2da54c987b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:56.306469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2bb5e-ad54-4462-a3b7-8db0a9f0d1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:56.307119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2bb5e-ad54-4462-a3b7-8db0a9f0d1ee None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:56.307396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2bb5e-ad54-4462-a3b7-8db0a9f0d1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:56.307665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2bb5e-ad54-4462-a3b7-8db0a9f0d1ee None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/users/1aa7307e2397438abfc4aef9b8d08b51/roles/6d101684527d45fabea90eeb60fc3e99` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:56.309585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2bb5e-ad54-4462-a3b7-8db0a9f0d1ee None admin] RBAC: Authorizing `identity:create_grant(project_id=6e0511b004c94f1bbed48dd4fd34cb6a, user_id=1aa7307e2397438abfc4aef9b8d08b51, role_id=6d101684527d45fabea90eeb60fc3e99)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:56.311022 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2bb5e-ad54-4462-a3b7-8db0a9f0d1ee None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:56.323321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 62/129] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:41:56 2022] PUT /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/users/1aa7307e2397438abfc4aef9b8d08b51/roles/6d101684527d45fabea90eeb60fc3e99 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:41:58.044754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178279f9-b020-4fb2-a996-1534b709b945 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:58.045273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178279f9-b020-4fb2-a996-1534b709b945 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:58.045273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178279f9-b020-4fb2-a996-1534b709b945 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:58.046618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 68/130] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:41:58.051793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab15dd-057a-482a-8055-f23c6cf97099 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:58.052229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab15dd-057a-482a-8055-f23c6cf97099 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:58.052377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab15dd-057a-482a-8055-f23c6cf97099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:58.070773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-8eab15dd-057a-482a-8055-f23c6cf97099 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:58.071981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8eab15dd-057a-482a-8055-f23c6cf97099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:58.137149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 63/131] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:41:58.407463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178279f9-b020-4fb2-a996-1534b709b945 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:58.451955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fc567-f765-4762-bc81-bfc1dd6b3675 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:58.452559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fc567-f765-4762-bc81-bfc1dd6b3675 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:58.452847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fc567-f765-4762-bc81-bfc1dd6b3675 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:58.454190 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 64/132] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:41:58.459097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6fea9-1f78-463e-b222-bb3a63d76ebd None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:58.459474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6fea9-1f78-463e-b222-bb3a63d76ebd None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:58.459856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6fea9-1f78-463e-b222-bb3a63d76ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:58.477293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6ed6fea9-1f78-463e-b222-bb3a63d76ebd None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:41:58.478480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6ed6fea9-1f78-463e-b222-bb3a63d76ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:41:58.489846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:58.511403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178279f9-b020-4fb2-a996-1534b709b945 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:58.543948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cafaa9-5a54-4ce2-b339-4627c3a87d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:58.544226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 65/133] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:41:58.544635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cafaa9-5a54-4ce2-b339-4627c3a87d3e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:58.544916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cafaa9-5a54-4ce2-b339-4627c3a87d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:58.545185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cafaa9-5a54-4ce2-b339-4627c3a87d3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:58.546339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cafaa9-5a54-4ce2-b339-4627c3a87d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:58.547783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cafaa9-5a54-4ce2-b339-4627c3a87d3e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:58.549501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 69/134] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:41:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:58.555387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6fea9-1f78-463e-b222-bb3a63d76ebd None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:58.557750 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:58.651408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6fea9-1f78-463e-b222-bb3a63d76ebd None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:58.683845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcc0d8-76c5-40d2-9e15-e34e54c50371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:58.684518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcc0d8-76c5-40d2-9e15-e34e54c50371 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:58.684795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcc0d8-76c5-40d2-9e15-e34e54c50371 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:58.685083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcc0d8-76c5-40d2-9e15-e34e54c50371 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:58.685732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcc0d8-76c5-40d2-9e15-e34e54c50371 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:58.687160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcc0d8-76c5-40d2-9e15-e34e54c50371 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:58.708927 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 66/135] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:41:58 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:41:58.820660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cafaa9-5a54-4ce2-b339-4627c3a87d3e None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:58.931610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cafaa9-5a54-4ce2-b339-4627c3a87d3e None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:58.966084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c639c-7836-4b6d-a139-14b7a2dc5c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:58.966727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c639c-7836-4b6d-a139-14b7a2dc5c90 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:58.967010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c639c-7836-4b6d-a139-14b7a2dc5c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:58.967282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c639c-7836-4b6d-a139-14b7a2dc5c90 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:58.979963 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c639c-7836-4b6d-a139-14b7a2dc5c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-895c639c-7836-4b6d-a139-14b7a2dc5c90 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:41:58.985298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:41:58.986145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:41:58.986909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 70/136] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:41:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:41:58.993054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcc0d8-76c5-40d2-9e15-e34e54c50371 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:59.059611 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:59.106872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcc0d8-76c5-40d2-9e15-e34e54c50371 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:59.144500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f99f-ecc1-4e0b-815d-e59703c1013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:59.145124 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f99f-ecc1-4e0b-815d-e59703c1013a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:59.145403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f99f-ecc1-4e0b-815d-e59703c1013a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:59.145674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f99f-ecc1-4e0b-815d-e59703c1013a None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:59.146348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536f99f-ecc1-4e0b-815d-e59703c1013a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:59.146650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536f99f-ecc1-4e0b-815d-e59703c1013a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:59.148708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536f99f-ecc1-4e0b-815d-e59703c1013a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:59.157758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 67/137] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:41:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:41:59.164036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c639c-7836-4b6d-a139-14b7a2dc5c90 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:59.176152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:59.274927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c639c-7836-4b6d-a139-14b7a2dc5c90 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:59.310551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55c60-e84a-4ad9-8d64-b527989286e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:59.311188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55c60-e84a-4ad9-8d64-b527989286e2 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:59.311465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55c60-e84a-4ad9-8d64-b527989286e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:59.311736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55c60-e84a-4ad9-8d64-b527989286e2 None admin] PATH_INFO: `/v3/users/1aa7307e2397438abfc4aef9b8d08b51` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:59.312897 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f55c60-e84a-4ad9-8d64-b527989286e2 None admin] RBAC: Authorizing `identity:get_user(user_id=1aa7307e2397438abfc4aef9b8d08b51)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:59.314365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f55c60-e84a-4ad9-8d64-b527989286e2 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:59.315983 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 71/138] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:41:59 2022] GET /identity/v3/users/1aa7307e2397438abfc4aef9b8d08b51 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:59.321792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f99f-ecc1-4e0b-815d-e59703c1013a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:59.432053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f99f-ecc1-4e0b-815d-e59703c1013a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:59.456758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:59.471369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf127ca-b617-4086-9715-ac73dd39690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:59.472009 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf127ca-b617-4086-9715-ac73dd39690c None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:59.472290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf127ca-b617-4086-9715-ac73dd39690c None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:59.472564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf127ca-b617-4086-9715-ac73dd39690c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:59.545540 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf127ca-b617-4086-9715-ac73dd39690c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:59.547050 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf127ca-b617-4086-9715-ac73dd39690c None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:59.651870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 68/139] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:41:59.657851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55c60-e84a-4ad9-8d64-b527989286e2 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:59.742881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:59.772737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55c60-e84a-4ad9-8d64-b527989286e2 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:59.812767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1132396-c7f4-405e-a681-d9f99b92b2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:59.813745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132396-c7f4-405e-a681-d9f99b92b2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:59.814286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132396-c7f4-405e-a681-d9f99b92b2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:59.814754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132396-c7f4-405e-a681-d9f99b92b2f9 None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:59.816112 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1132396-c7f4-405e-a681-d9f99b92b2f9 None admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:59.817763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1132396-c7f4-405e-a681-d9f99b92b2f9 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:59.819658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 72/140] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:41:59 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:41:59.825644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf127ca-b617-4086-9715-ac73dd39690c None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:41:59.847567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:41:59.932928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf127ca-b617-4086-9715-ac73dd39690c None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:41:59.969578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe5b44-8e38-4455-bbb0-b8c2a8155aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:41:59.970208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe5b44-8e38-4455-bbb0-b8c2a8155aea None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:41:59.970491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe5b44-8e38-4455-bbb0-b8c2a8155aea None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:41:59.970766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe5b44-8e38-4455-bbb0-b8c2a8155aea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:41:59.971510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe5b44-8e38-4455-bbb0-b8c2a8155aea None admin] RBAC: Adding query filter params (role.id=6d101684527d45fabea90eeb60fc3e99, scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a, user.id=1aa7307e2397438abfc4aef9b8d08b51) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:41:59.971813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe5b44-8e38-4455-bbb0-b8c2a8155aea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:41:59.973390 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe5b44-8e38-4455-bbb0-b8c2a8155aea None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:41:59.980339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 69/141] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:41:59 2022] GET /identity/v3/role_assignments?user.id=1aa7307e2397438abfc4aef9b8d08b51&scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a&role.id=6d101684527d45fabea90eeb60fc3e99 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:01.042304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b1fa7-8291-49ac-81fb-a1d10a8b3323 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.042911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b1fa7-8291-49ac-81fb-a1d10a8b3323 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.042911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b1fa7-8291-49ac-81fb-a1d10a8b3323 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.044249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 73/142] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:01.049421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a755d-9099-4354-a154-4f9bbc8253bd None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.049822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a755d-9099-4354-a154-4f9bbc8253bd None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.049977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a755d-9099-4354-a154-4f9bbc8253bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.069871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-705a755d-9099-4354-a154-4f9bbc8253bd None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:01.071391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-705a755d-9099-4354-a154-4f9bbc8253bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:01.145241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 70/143] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:01.187160 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aade84-353b-448f-96ae-d2acb75a0cca None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.187480 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aade84-353b-448f-96ae-d2acb75a0cca None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.187704 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aade84-353b-448f-96ae-d2acb75a0cca None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.189080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 74/144] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:01.194305 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7b42-9219-45bf-a774-93165d087377 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.194636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7b42-9219-45bf-a774-93165d087377 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.194870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7b42-9219-45bf-a774-93165d087377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.213036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-17da7b42-9219-45bf-a774-93165d087377 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:01.214221 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-17da7b42-9219-45bf-a774-93165d087377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:01.224282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:01.249264 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83dd17-c099-46ba-8786-79f7106a322e None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.249640 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83dd17-c099-46ba-8786-79f7106a322e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.249919 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83dd17-c099-46ba-8786-79f7106a322e None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.251236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 75/145] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:42:01.256079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d1b02-bd95-438a-aae1-3961ccbc8811 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.256404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d1b02-bd95-438a-aae1-3961ccbc8811 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.256674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d1b02-bd95-438a-aae1-3961ccbc8811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.276736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-3f4d1b02-bd95-438a-aae1-3961ccbc8811 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:01.277931 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3f4d1b02-bd95-438a-aae1-3961ccbc8811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:01.287416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 71/146] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:01.353878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 76/147] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:01.419556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da7b42-9219-45bf-a774-93165d087377 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:01.519493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da7b42-9219-45bf-a774-93165d087377 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:01.553987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007072c4-3fda-4843-a860-79e2e08438e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:01.554615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007072c4-3fda-4843-a860-79e2e08438e1 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.554903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007072c4-3fda-4843-a860-79e2e08438e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.555172 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007072c4-3fda-4843-a860-79e2e08438e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.560886 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d1b02-bd95-438a-aae1-3961ccbc8811 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:01.566706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007072c4-3fda-4843-a860-79e2e08438e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-007072c4-3fda-4843-a860-79e2e08438e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:01.572922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:01.573800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:01.574807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 72/148] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:42:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:01.580057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007072c4-3fda-4843-a860-79e2e08438e1 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:01.585393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:01.642708 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:01.668159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d1b02-bd95-438a-aae1-3961ccbc8811 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:01.684486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007072c4-3fda-4843-a860-79e2e08438e1 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:01.701220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481675b-d6c2-437a-a4e5-20ae04f52e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:01.701950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481675b-d6c2-437a-a4e5-20ae04f52e5f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.702228 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481675b-d6c2-437a-a4e5-20ae04f52e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.702495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481675b-d6c2-437a-a4e5-20ae04f52e5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.713026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481675b-d6c2-437a-a4e5-20ae04f52e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-f481675b-d6c2-437a-a4e5-20ae04f52e5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:01.717774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:01.718636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:01.719685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 77/149] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:42:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:01.720656 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc717602-2bc9-4e7e-85e1-226598298e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:01.720754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481675b-d6c2-437a-a4e5-20ae04f52e5f None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:01.720840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc717602-2bc9-4e7e-85e1-226598298e52 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.720840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc717602-2bc9-4e7e-85e1-226598298e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.720840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc717602-2bc9-4e7e-85e1-226598298e52 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.721330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc717602-2bc9-4e7e-85e1-226598298e52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:01.721630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc717602-2bc9-4e7e-85e1-226598298e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:01.723020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc717602-2bc9-4e7e-85e1-226598298e52 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:01.735674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 73/150] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:01.735674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc717602-2bc9-4e7e-85e1-226598298e52 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:01.799045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:01.822940 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481675b-d6c2-437a-a4e5-20ae04f52e5f None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:01.833441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc717602-2bc9-4e7e-85e1-226598298e52 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:01.856931 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39b2a7-2858-4c05-8787-421434315492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:01.857543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39b2a7-2858-4c05-8787-421434315492 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.857816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39b2a7-2858-4c05-8787-421434315492 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.858084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39b2a7-2858-4c05-8787-421434315492 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.859178 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39b2a7-2858-4c05-8787-421434315492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:01.860642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39b2a7-2858-4c05-8787-421434315492 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:01.862189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 78/151] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:01.862928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39b2a7-2858-4c05-8787-421434315492 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:01.865452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:01.867342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9b875-49d9-4ab5-986e-aed6d1e167c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:01.867707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9b875-49d9-4ab5-986e-aed6d1e167c1 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.868062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9b875-49d9-4ab5-986e-aed6d1e167c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.868336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9b875-49d9-4ab5-986e-aed6d1e167c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:01.869004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9b875-49d9-4ab5-986e-aed6d1e167c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:01.869298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9b875-49d9-4ab5-986e-aed6d1e167c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:01.870654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9b875-49d9-4ab5-986e-aed6d1e167c1 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:01.879279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 74/152] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:01.879849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9b875-49d9-4ab5-986e-aed6d1e167c1 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:01.965526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39b2a7-2858-4c05-8787-421434315492 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:01.987698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9b875-49d9-4ab5-986e-aed6d1e167c1 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:01.998138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc606525-c66f-44e6-8087-6487ddd69405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:01.998764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606525-c66f-44e6-8087-6487ddd69405 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:01.999036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606525-c66f-44e6-8087-6487ddd69405 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:01.999297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606525-c66f-44e6-8087-6487ddd69405 None admin] PATH_INFO: `/v3/users/97ea986d150b4f2b84a3eb00a82a97e5` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:02.013142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc606525-c66f-44e6-8087-6487ddd69405 None admin] RBAC: Authorizing `identity:get_user(user_id=97ea986d150b4f2b84a3eb00a82a97e5)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:02.013754 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:02.014610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc606525-c66f-44e6-8087-6487ddd69405 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:02.016255 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 79/153] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:01 2022] GET /identity/v3/users/97ea986d150b4f2b84a3eb00a82a97e5 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:02.021627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc606525-c66f-44e6-8087-6487ddd69405 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:02.023177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f35fb-4236-4fc4-8575-aba3d8865113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:02.023849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f35fb-4236-4fc4-8575-aba3d8865113 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:02.024195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f35fb-4236-4fc4-8575-aba3d8865113 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:02.024504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f35fb-4236-4fc4-8575-aba3d8865113 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:02.025181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f35fb-4236-4fc4-8575-aba3d8865113 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:02.026647 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f35fb-4236-4fc4-8575-aba3d8865113 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:02.047667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 75/154] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:42:01 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:42:02.048407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f35fb-4236-4fc4-8575-aba3d8865113 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:02.092164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:02.136356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc606525-c66f-44e6-8087-6487ddd69405 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:02.169382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f35fb-4236-4fc4-8575-aba3d8865113 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:02.173747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a6e89-e19f-4eaf-b918-65bf8622655a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:02.174398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a6e89-e19f-4eaf-b918-65bf8622655a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:02.174679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a6e89-e19f-4eaf-b918-65bf8622655a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:02.174957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a6e89-e19f-4eaf-b918-65bf8622655a None admin] PATH_INFO: `/v3/users/97ea986d150b4f2b84a3eb00a82a97e5` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:02.176197 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a6e89-e19f-4eaf-b918-65bf8622655a None admin] RBAC: Authorizing `identity:get_user(user_id=97ea986d150b4f2b84a3eb00a82a97e5)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:02.178845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a6e89-e19f-4eaf-b918-65bf8622655a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:02.181323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 80/155] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:02 2022] GET /identity/v3/users/97ea986d150b4f2b84a3eb00a82a97e5 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:02.187043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a6e89-e19f-4eaf-b918-65bf8622655a None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:02.211907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ef476-91e0-40ee-8be8-5a3c48951643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:02.212666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ef476-91e0-40ee-8be8-5a3c48951643 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:02.212990 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ef476-91e0-40ee-8be8-5a3c48951643 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:02.213394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ef476-91e0-40ee-8be8-5a3c48951643 None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:02.214603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ef476-91e0-40ee-8be8-5a3c48951643 None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:02.216399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ef476-91e0-40ee-8be8-5a3c48951643 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:02.218181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 76/156] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:02 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:02.223921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ef476-91e0-40ee-8be8-5a3c48951643 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:02.285492 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:02.302816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a6e89-e19f-4eaf-b918-65bf8622655a None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:02.337758 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:02.348019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4134ccb-bc55-41fe-890c-1e55f4f1eb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:02.349238 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4134ccb-bc55-41fe-890c-1e55f4f1eb3a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:02.349724 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4134ccb-bc55-41fe-890c-1e55f4f1eb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:02.350936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4134ccb-bc55-41fe-890c-1e55f4f1eb3a None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:02.352177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4134ccb-bc55-41fe-890c-1e55f4f1eb3a None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:02.353634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4134ccb-bc55-41fe-890c-1e55f4f1eb3a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:02.355079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ef476-91e0-40ee-8be8-5a3c48951643 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:02.355597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 81/157] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:02 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:02.363521 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4134ccb-bc55-41fe-890c-1e55f4f1eb3a None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:02.404052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d697201-8bab-47d4-9bfe-b469f69fe4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:02.404771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d697201-8bab-47d4-9bfe-b469f69fe4bb None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:02.405106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d697201-8bab-47d4-9bfe-b469f69fe4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:02.405375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d697201-8bab-47d4-9bfe-b469f69fe4bb None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/users/97ea986d150b4f2b84a3eb00a82a97e5/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:02.409225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d697201-8bab-47d4-9bfe-b469f69fe4bb None admin] RBAC: Authorizing `identity:create_grant(project_id=8a5d8b8f16a64419ab27d85700e9bdb5, user_id=97ea986d150b4f2b84a3eb00a82a97e5, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:02.410674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d697201-8bab-47d4-9bfe-b469f69fe4bb None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:02.425191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 77/158] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:02 2022] PUT /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/users/97ea986d150b4f2b84a3eb00a82a97e5/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:42:02.492648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4134ccb-bc55-41fe-890c-1e55f4f1eb3a None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:02.531688 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00531eb-8bba-4a5d-acfe-4508da15bd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:02.532405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00531eb-8bba-4a5d-acfe-4508da15bd5c None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:02.532679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00531eb-8bba-4a5d-acfe-4508da15bd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:02.532944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00531eb-8bba-4a5d-acfe-4508da15bd5c None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/users/97ea986d150b4f2b84a3eb00a82a97e5/roles/1cc2ea78f84c4491967839cfead3fd26` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:02.535195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00531eb-8bba-4a5d-acfe-4508da15bd5c None admin] RBAC: Authorizing `identity:create_grant(project_id=8a5d8b8f16a64419ab27d85700e9bdb5, user_id=97ea986d150b4f2b84a3eb00a82a97e5, role_id=1cc2ea78f84c4491967839cfead3fd26)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:02.536818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00531eb-8bba-4a5d-acfe-4508da15bd5c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:02.552586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 82/159] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:02 2022] PUT /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/users/97ea986d150b4f2b84a3eb00a82a97e5/roles/1cc2ea78f84c4491967839cfead3fd26 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:04.366212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4bc1e-22b6-41ea-a395-67002bc7b6e3 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:04.366743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4bc1e-22b6-41ea-a395-67002bc7b6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:04.366743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4bc1e-22b6-41ea-a395-67002bc7b6e3 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:04.368358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 78/160] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:04.374292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffee686-68a8-42d3-b51d-5cd7f9cc9e10 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:04.374817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffee686-68a8-42d3-b51d-5cd7f9cc9e10 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:04.374817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffee686-68a8-42d3-b51d-5cd7f9cc9e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:04.394888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0ffee686-68a8-42d3-b51d-5cd7f9cc9e10 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:04.396105 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0ffee686-68a8-42d3-b51d-5cd7f9cc9e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:04.447796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:04.469628 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 83/161] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:04.484372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda23de6-5cb8-4f4e-b785-1a3c47bd3f24 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:04.484650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda23de6-5cb8-4f4e-b785-1a3c47bd3f24 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:04.484917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda23de6-5cb8-4f4e-b785-1a3c47bd3f24 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:04.486515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 79/162] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:04.492758 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff733-89f3-4a74-bd60-74c816e0854a None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:04.493084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff733-89f3-4a74-bd60-74c816e0854a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:04.493312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff733-89f3-4a74-bd60-74c816e0854a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:04.512789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-3a4ff733-89f3-4a74-bd60-74c816e0854a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:04.513971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3a4ff733-89f3-4a74-bd60-74c816e0854a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:04.587979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 84/163] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:04.685768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a05f8c-63fa-41be-9ac7-1a47dd1100d7 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:04.685954 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a05f8c-63fa-41be-9ac7-1a47dd1100d7 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:04.686220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a05f8c-63fa-41be-9ac7-1a47dd1100d7 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:04.687955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 80/164] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:04.693660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30421c-21b4-42fc-8497-337b5f88bead None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:04.694585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30421c-21b4-42fc-8497-337b5f88bead None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:04.694806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30421c-21b4-42fc-8497-337b5f88bead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:04.713099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4d30421c-21b4-42fc-8497-337b5f88bead None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:04.714281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4d30421c-21b4-42fc-8497-337b5f88bead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:04.743698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a05f8c-63fa-41be-9ac7-1a47dd1100d7 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:04.785858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 85/165] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:04.799733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b8089-7cde-41ac-9d83-4f17d6b9bd8d None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:04.800047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b8089-7cde-41ac-9d83-4f17d6b9bd8d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:04.800326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b8089-7cde-41ac-9d83-4f17d6b9bd8d None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:04.801888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 86/166] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:42:04.809144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe6c23-4039-4b96-b02e-58172337f6ca None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:04.809431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe6c23-4039-4b96-b02e-58172337f6ca None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:04.809707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe6c23-4039-4b96-b02e-58172337f6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:04.835020 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-95fe6c23-4039-4b96-b02e-58172337f6ca None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:04.836070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-95fe6c23-4039-4b96-b02e-58172337f6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:04.855397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a05f8c-63fa-41be-9ac7-1a47dd1100d7 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:04.871642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:04.884617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:04.896765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1f550-3b39-4a1c-acfe-2c00f67adee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:04.897608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1f550-3b39-4a1c-acfe-2c00f67adee8 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:04.897944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1f550-3b39-4a1c-acfe-2c00f67adee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:04.898303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1f550-3b39-4a1c-acfe-2c00f67adee8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:04.908975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 87/167] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:04.909309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1f550-3b39-4a1c-acfe-2c00f67adee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:04.909802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe6c23-4039-4b96-b02e-58172337f6ca None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-00c1f550-3b39-4a1c-acfe-2c00f67adee8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:04.914802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:04.915774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:04.916881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 81/168] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:04.924438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1f550-3b39-4a1c-acfe-2c00f67adee8 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.036447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe6c23-4039-4b96-b02e-58172337f6ca None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.049559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1f550-3b39-4a1c-acfe-2c00f67adee8 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.080211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f14b9-30cf-4e8d-9055-9534db4d47ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.081005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f14b9-30cf-4e8d-9055-9534db4d47ba None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.081381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f14b9-30cf-4e8d-9055-9534db4d47ba None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.081768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f14b9-30cf-4e8d-9055-9534db4d47ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.083380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f14b9-30cf-4e8d-9055-9534db4d47ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:05.085735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f14b9-30cf-4e8d-9055-9534db4d47ba None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:05.087566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 88/169] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:05.089131 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0549f78-61ce-4f49-89b8-490f130e2bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.090026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0549f78-61ce-4f49-89b8-490f130e2bfa None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.090301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0549f78-61ce-4f49-89b8-490f130e2bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.090570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0549f78-61ce-4f49-89b8-490f130e2bfa None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.091234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0549f78-61ce-4f49-89b8-490f130e2bfa None admin] RBAC: Adding query filter params (name=member) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:05.091530 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0549f78-61ce-4f49-89b8-490f130e2bfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:05.092721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f14b9-30cf-4e8d-9055-9534db4d47ba None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.092923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0549f78-61ce-4f49-89b8-490f130e2bfa None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:05.102267 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 82/170] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:05.104317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0549f78-61ce-4f49-89b8-490f130e2bfa None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.140331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:05.141746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:05.199257 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f14b9-30cf-4e8d-9055-9534db4d47ba None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.210853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0549f78-61ce-4f49-89b8-490f130e2bfa None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.233613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2438-126d-4cfa-af98-9844ed3251de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.234236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2438-126d-4cfa-af98-9844ed3251de None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.234516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2438-126d-4cfa-af98-9844ed3251de None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.234786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2438-126d-4cfa-af98-9844ed3251de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.244160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b57c3d-3711-45c4-9697-65386dbcc0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.244762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b57c3d-3711-45c4-9697-65386dbcc0b5 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.245033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b57c3d-3711-45c4-9697-65386dbcc0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.245170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e2438-126d-4cfa-af98-9844ed3251de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:05.245301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b57c3d-3711-45c4-9697-65386dbcc0b5 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.245940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b57c3d-3711-45c4-9697-65386dbcc0b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:05.247372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b57c3d-3711-45c4-9697-65386dbcc0b5 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-471e2438-126d-4cfa-af98-9844ed3251de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:05.250003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:05.250869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 89/171] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:42:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:05.252568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2438-126d-4cfa-af98-9844ed3251de None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.269284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 83/172] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:42:05 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:42:05.277776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b57c3d-3711-45c4-9697-65386dbcc0b5 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.368027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2438-126d-4cfa-af98-9844ed3251de None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.371747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:05.403828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:05.414206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328256ea-81c5-41a7-bd29-75b40ccfdece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.414868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328256ea-81c5-41a7-bd29-75b40ccfdece None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.415137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328256ea-81c5-41a7-bd29-75b40ccfdece None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.415411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328256ea-81c5-41a7-bd29-75b40ccfdece None admin] PATH_INFO: `/v3/users/88fc56cff73e41bfa79becb4c3b65f21` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.416228 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b57c3d-3711-45c4-9697-65386dbcc0b5 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.432220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328256ea-81c5-41a7-bd29-75b40ccfdece None admin] RBAC: Authorizing `identity:get_user(user_id=88fc56cff73e41bfa79becb4c3b65f21)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:05.434217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328256ea-81c5-41a7-bd29-75b40ccfdece None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:05.436419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 90/173] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:05 2022] GET /identity/v3/users/88fc56cff73e41bfa79becb4c3b65f21 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:05.437526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328256ea-81c5-41a7-bd29-75b40ccfdece None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.460838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf407e61-7e15-42db-b235-f976c600150b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.461481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf407e61-7e15-42db-b235-f976c600150b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.461762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf407e61-7e15-42db-b235-f976c600150b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.462034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf407e61-7e15-42db-b235-f976c600150b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.476827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf407e61-7e15-42db-b235-f976c600150b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-cf407e61-7e15-42db-b235-f976c600150b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:05.482252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:05.483164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 84/174] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:42:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:05.484568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf407e61-7e15-42db-b235-f976c600150b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.566425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328256ea-81c5-41a7-bd29-75b40ccfdece None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.609202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f1d11-38fd-413a-8c16-1ea765e7135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.609742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f1d11-38fd-413a-8c16-1ea765e7135b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.610018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f1d11-38fd-413a-8c16-1ea765e7135b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.610286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f1d11-38fd-413a-8c16-1ea765e7135b None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.610541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf407e61-7e15-42db-b235-f976c600150b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.610955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f1d11-38fd-413a-8c16-1ea765e7135b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:05.611251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f1d11-38fd-413a-8c16-1ea765e7135b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:05.612664 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f1d11-38fd-413a-8c16-1ea765e7135b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:05.623021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 91/175] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:05.624297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f1d11-38fd-413a-8c16-1ea765e7135b None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.659472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:05.661720 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28099e-c1b3-464f-bfa9-2f347a952aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.662179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28099e-c1b3-464f-bfa9-2f347a952aef None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.662986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28099e-c1b3-464f-bfa9-2f347a952aef None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.663385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28099e-c1b3-464f-bfa9-2f347a952aef None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.665510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28099e-c1b3-464f-bfa9-2f347a952aef None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:05.666976 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28099e-c1b3-464f-bfa9-2f347a952aef None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:05.668822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 85/176] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:05 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:05.669430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28099e-c1b3-464f-bfa9-2f347a952aef None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.695686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:05.750229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f1d11-38fd-413a-8c16-1ea765e7135b None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.779428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28099e-c1b3-464f-bfa9-2f347a952aef None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.789181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a65f6-9021-4acc-aab8-05025f7fd318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.790136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a65f6-9021-4acc-aab8-05025f7fd318 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.790650 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a65f6-9021-4acc-aab8-05025f7fd318 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.791164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a65f6-9021-4acc-aab8-05025f7fd318 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.792076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a65f6-9021-4acc-aab8-05025f7fd318 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:05.792597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a65f6-9021-4acc-aab8-05025f7fd318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:05.794446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a65f6-9021-4acc-aab8-05025f7fd318 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:05.804212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 92/177] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:05.809784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a65f6-9021-4acc-aab8-05025f7fd318 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.815714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bd7db-12ae-49c0-ab15-9c92e95c9bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.816403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bd7db-12ae-49c0-ab15-9c92e95c9bbd None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.816682 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bd7db-12ae-49c0-ab15-9c92e95c9bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.816951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bd7db-12ae-49c0-ab15-9c92e95c9bbd None admin] PATH_INFO: `/v3/users/97ea986d150b4f2b84a3eb00a82a97e5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.818092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bd7db-12ae-49c0-ab15-9c92e95c9bbd None admin] RBAC: Authorizing `identity:get_user(user_id=97ea986d150b4f2b84a3eb00a82a97e5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:05.819658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bd7db-12ae-49c0-ab15-9c92e95c9bbd None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:05.821627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 86/178] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:05 2022] GET /identity/v3/users/97ea986d150b4f2b84a3eb00a82a97e5 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:05.822409 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bd7db-12ae-49c0-ab15-9c92e95c9bbd None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:05.921669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:05.938524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a65f6-9021-4acc-aab8-05025f7fd318 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.942074 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bd7db-12ae-49c0-ab15-9c92e95c9bbd None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:05.960280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:05.981600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c43019-eb11-4b88-bea4-7bf11e7be0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.982420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43019-eb11-4b88-bea4-7bf11e7be0c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.982795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43019-eb11-4b88-bea4-7bf11e7be0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.983192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43019-eb11-4b88-bea4-7bf11e7be0c6 None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/users/88fc56cff73e41bfa79becb4c3b65f21/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.984991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4bbd-813e-4392-9279-a1513fb179fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:05.985350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c43019-eb11-4b88-bea4-7bf11e7be0c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a5d8b8f16a64419ab27d85700e9bdb5, user_id=88fc56cff73e41bfa79becb4c3b65f21, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:05.985959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4bbd-813e-4392-9279-a1513fb179fb None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:05.986382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4bbd-813e-4392-9279-a1513fb179fb None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:05.986787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4bbd-813e-4392-9279-a1513fb179fb None admin] PATH_INFO: `/v3/users/97ea986d150b4f2b84a3eb00a82a97e5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:05.987190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c43019-eb11-4b88-bea4-7bf11e7be0c6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:05.988252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b4bbd-813e-4392-9279-a1513fb179fb None admin] RBAC: Authorizing `identity:get_user(user_id=97ea986d150b4f2b84a3eb00a82a97e5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:05.990015 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b4bbd-813e-4392-9279-a1513fb179fb None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:05.992690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 87/179] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:05 2022] GET /identity/v3/users/97ea986d150b4f2b84a3eb00a82a97e5 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:05.996898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4bbd-813e-4392-9279-a1513fb179fb None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:06.005344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 93/180] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:05 2022] PUT /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/users/88fc56cff73e41bfa79becb4c3b65f21/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:06.006547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c43019-eb11-4b88-bea4-7bf11e7be0c6 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:06.137292 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4bbd-813e-4392-9279-a1513fb179fb None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:06.183463 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2949b07-8077-4f27-84e4-4847b55ed259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:06.185504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2949b07-8077-4f27-84e4-4847b55ed259 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:06.186831 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2949b07-8077-4f27-84e4-4847b55ed259 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:06.187451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2949b07-8077-4f27-84e4-4847b55ed259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:06.222615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:06.233727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c43019-eb11-4b88-bea4-7bf11e7be0c6 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:06.282126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2949b07-8077-4f27-84e4-4847b55ed259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:06.283858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2949b07-8077-4f27-84e4-4847b55ed259 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:06.315298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f7f12-af08-418d-b177-85fadadefb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:06.316305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f7f12-af08-418d-b177-85fadadefb4b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:06.316677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f7f12-af08-418d-b177-85fadadefb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:06.317045 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f7f12-af08-418d-b177-85fadadefb4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:06.336843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:06.408347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f7f12-af08-418d-b177-85fadadefb4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:06.410358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f7f12-af08-418d-b177-85fadadefb4b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:06.442532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 88/181] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:06.448075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2949b07-8077-4f27-84e4-4847b55ed259 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:06.530835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 94/182] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:06.536758 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f7f12-af08-418d-b177-85fadadefb4b None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:06.571694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:06.625630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:06.664201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2949b07-8077-4f27-84e4-4847b55ed259 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:06.706384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f7f12-af08-418d-b177-85fadadefb4b None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:06.742522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22406b9a-6804-498d-9edf-d14e0988a50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:06.743163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22406b9a-6804-498d-9edf-d14e0988a50b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:06.743434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22406b9a-6804-498d-9edf-d14e0988a50b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:06.743702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22406b9a-6804-498d-9edf-d14e0988a50b None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:06.746976 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400902ba-646a-437a-ab64-819e0a98f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:06.747780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400902ba-646a-437a-ab64-819e0a98f8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:06.748259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400902ba-646a-437a-ab64-819e0a98f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:06.748584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22406b9a-6804-498d-9edf-d14e0988a50b None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:06.748815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400902ba-646a-437a-ab64-819e0a98f8c9 None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:06.750052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22406b9a-6804-498d-9edf-d14e0988a50b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:06.750270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400902ba-646a-437a-ab64-819e0a98f8c9 None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:06.751850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400902ba-646a-437a-ab64-819e0a98f8c9 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:06.752037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 89/183] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:06 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:06.754036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 95/184] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:06 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:06.763317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22406b9a-6804-498d-9edf-d14e0988a50b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:06.764550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400902ba-646a-437a-ab64-819e0a98f8c9 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:06.874862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400902ba-646a-437a-ab64-819e0a98f8c9 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:06.900119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:06.914287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1676b-3766-40b6-a3aa-317ee8fec5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:06.915281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1676b-3766-40b6-a3aa-317ee8fec5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:06.915923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1676b-3766-40b6-a3aa-317ee8fec5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:06.916336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1676b-3766-40b6-a3aa-317ee8fec5c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:06.917198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1676b-3766-40b6-a3aa-317ee8fec5c6 None admin] RBAC: Adding query filter params (role.id=1cc2ea78f84c4491967839cfead3fd26, scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5, user.id=97ea986d150b4f2b84a3eb00a82a97e5) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:06.917642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1676b-3766-40b6-a3aa-317ee8fec5c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:06.919297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1676b-3766-40b6-a3aa-317ee8fec5c6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:06.926306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 96/185] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:42:06 2022] GET /identity/v3/role_assignments?user.id=97ea986d150b4f2b84a3eb00a82a97e5&scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5&role.id=1cc2ea78f84c4491967839cfead3fd26 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:06.996907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22406b9a-6804-498d-9edf-d14e0988a50b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:07.039826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:07.051910 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b8af9-f1eb-46a9-a324-f7982f727719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:07.052522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b8af9-f1eb-46a9-a324-f7982f727719 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:07.052794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b8af9-f1eb-46a9-a324-f7982f727719 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:07.053084 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b8af9-f1eb-46a9-a324-f7982f727719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:07.054531 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b8af9-f1eb-46a9-a324-f7982f727719 None admin] RBAC: Adding query filter params (role.id=45b67ba9922c4b16baae3930d3de1b6c, scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5, user.id=97ea986d150b4f2b84a3eb00a82a97e5) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:07.054849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b8af9-f1eb-46a9-a324-f7982f727719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:07.056322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b8af9-f1eb-46a9-a324-f7982f727719 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:07.067366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 90/186] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:42:06 2022] GET /identity/v3/role_assignments?user.id=97ea986d150b4f2b84a3eb00a82a97e5&scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5&role.id=45b67ba9922c4b16baae3930d3de1b6c => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:07.817374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1b6a-94db-4b0b-bbea-7c525973da64 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:07.818365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1b6a-94db-4b0b-bbea-7c525973da64 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:07.818908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1b6a-94db-4b0b-bbea-7c525973da64 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:07.820481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 97/187] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:07.822933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daeac2-b63b-4db9-a063-49e3b321d145 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:07.823330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daeac2-b63b-4db9-a063-49e3b321d145 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:07.823486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daeac2-b63b-4db9-a063-49e3b321d145 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:07.824799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 91/188] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:07.825632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7960217-eb37-4d6e-8832-0d1d5ae170ce None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:07.826129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7960217-eb37-4d6e-8832-0d1d5ae170ce None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:07.826586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7960217-eb37-4d6e-8832-0d1d5ae170ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:07.829439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efaf8a-210f-459d-87b7-02982db6c2dc None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:07.829713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efaf8a-210f-459d-87b7-02982db6c2dc None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:07.830009 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efaf8a-210f-459d-87b7-02982db6c2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:07.848100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-15efaf8a-210f-459d-87b7-02982db6c2dc None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:07.848816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-a7960217-eb37-4d6e-8832-0d1d5ae170ce None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:07.849242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-15efaf8a-210f-459d-87b7-02982db6c2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:07.851174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a7960217-eb37-4d6e-8832-0d1d5ae170ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:07.922187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 92/189] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:07.941281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 98/190] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:08.201507 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efaf8a-210f-459d-87b7-02982db6c2dc None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:08.210908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7960217-eb37-4d6e-8832-0d1d5ae170ce None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:08.308596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efaf8a-210f-459d-87b7-02982db6c2dc None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:08.325334 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:08.332460 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7960217-eb37-4d6e-8832-0d1d5ae170ce None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:08.347631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691bc45-ccad-4267-8f8c-653d8a2478ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:08.348306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691bc45-ccad-4267-8f8c-653d8a2478ab None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:08.348581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691bc45-ccad-4267-8f8c-653d8a2478ab None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:08.348853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691bc45-ccad-4267-8f8c-653d8a2478ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:08.349689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:08.361482 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691bc45-ccad-4267-8f8c-653d8a2478ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-6691bc45-ccad-4267-8f8c-653d8a2478ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:08.366894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:08.367774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:08.368600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 93/191] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:08.375107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691bc45-ccad-4267-8f8c-653d8a2478ab None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:08.378544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344179d-d68e-4e60-8de7-70cb9a115eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:08.379312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344179d-d68e-4e60-8de7-70cb9a115eef None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:08.379657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344179d-d68e-4e60-8de7-70cb9a115eef None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:08.379961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344179d-d68e-4e60-8de7-70cb9a115eef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:08.381068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344179d-d68e-4e60-8de7-70cb9a115eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:08.382878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344179d-d68e-4e60-8de7-70cb9a115eef None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:08.384494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 99/192] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:08.390093 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344179d-d68e-4e60-8de7-70cb9a115eef None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:08.487850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691bc45-ccad-4267-8f8c-653d8a2478ab None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:08.501794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344179d-d68e-4e60-8de7-70cb9a115eef None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:08.524575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac04207-2f4b-44f2-a606-767c7e8cd6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:08.525200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac04207-2f4b-44f2-a606-767c7e8cd6dc None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:08.525474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac04207-2f4b-44f2-a606-767c7e8cd6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:08.525747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac04207-2f4b-44f2-a606-767c7e8cd6dc None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:08.526416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac04207-2f4b-44f2-a606-767c7e8cd6dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:08.526718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac04207-2f4b-44f2-a606-767c7e8cd6dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:08.528321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac04207-2f4b-44f2-a606-767c7e8cd6dc None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:08.537187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 94/193] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:08.539641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797feb7-f8eb-42d8-a35c-8eb9d955f550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:08.540287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797feb7-f8eb-42d8-a35c-8eb9d955f550 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:08.540564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797feb7-f8eb-42d8-a35c-8eb9d955f550 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:08.540832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797feb7-f8eb-42d8-a35c-8eb9d955f550 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:08.541652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797feb7-f8eb-42d8-a35c-8eb9d955f550 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:08.542506 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac04207-2f4b-44f2-a606-767c7e8cd6dc None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:08.542984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797feb7-f8eb-42d8-a35c-8eb9d955f550 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:08.563374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 100/194] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:42:08 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:42:08.565282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506364cf-9692-4940-953d-9035c912ce67 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:08.566050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506364cf-9692-4940-953d-9035c912ce67 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:08.566339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506364cf-9692-4940-953d-9035c912ce67 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:08.567774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 101/195] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:08.574177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579d79e-8e16-4f85-9d13-69d68a1785c5 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:08.574460 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579d79e-8e16-4f85-9d13-69d68a1785c5 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:08.574730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579d79e-8e16-4f85-9d13-69d68a1785c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:08.589099 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:08.594005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e579d79e-8e16-4f85-9d13-69d68a1785c5 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:08.595233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e579d79e-8e16-4f85-9d13-69d68a1785c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:08.626342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:08.665088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac04207-2f4b-44f2-a606-767c7e8cd6dc None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:08.679927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 102/196] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:08.704826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cfd7b-368d-4505-9bc7-b8907357b052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:08.705631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cfd7b-368d-4505-9bc7-b8907357b052 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:08.706003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cfd7b-368d-4505-9bc7-b8907357b052 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:08.706372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cfd7b-368d-4505-9bc7-b8907357b052 None admin] PATH_INFO: `/v3/users/5dbd9560ac48432a903adec3848c9638` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:08.721964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44cfd7b-368d-4505-9bc7-b8907357b052 None admin] RBAC: Authorizing `identity:get_user(user_id=5dbd9560ac48432a903adec3848c9638)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:08.723617 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44cfd7b-368d-4505-9bc7-b8907357b052 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:08.726371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 95/197] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:08 2022] GET /identity/v3/users/5dbd9560ac48432a903adec3848c9638 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:08.731822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579d79e-8e16-4f85-9d13-69d68a1785c5 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:08.836524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579d79e-8e16-4f85-9d13-69d68a1785c5 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:08.873145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ac712-685e-431a-af11-5563b06f4cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:08.873784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ac712-685e-431a-af11-5563b06f4cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:08.874121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ac712-685e-431a-af11-5563b06f4cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:08.874393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ac712-685e-431a-af11-5563b06f4cc4 None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:08.875561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ac712-685e-431a-af11-5563b06f4cc4 None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:08.877051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ac712-685e-431a-af11-5563b06f4cc4 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:08.878689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 103/198] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:08 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:08.884789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cfd7b-368d-4505-9bc7-b8907357b052 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:08.950910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ac712-685e-431a-af11-5563b06f4cc4 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:08.988309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:09.006024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cfd7b-368d-4505-9bc7-b8907357b052 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:09.024481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:09.044520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52370184-1e86-4e7f-b0a4-0ed4215f2da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:09.045214 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52370184-1e86-4e7f-b0a4-0ed4215f2da2 None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:09.045766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52370184-1e86-4e7f-b0a4-0ed4215f2da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:09.045873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52370184-1e86-4e7f-b0a4-0ed4215f2da2 None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/users/5dbd9560ac48432a903adec3848c9638/roles/6d101684527d45fabea90eeb60fc3e99` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:09.047816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52370184-1e86-4e7f-b0a4-0ed4215f2da2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a5d8b8f16a64419ab27d85700e9bdb5, user_id=5dbd9560ac48432a903adec3848c9638, role_id=6d101684527d45fabea90eeb60fc3e99)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:09.049543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52370184-1e86-4e7f-b0a4-0ed4215f2da2 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:09.068157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 96/199] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:08 2022] PUT /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/users/5dbd9560ac48432a903adec3848c9638/roles/6d101684527d45fabea90eeb60fc3e99 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:09.071631 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ac712-685e-431a-af11-5563b06f4cc4 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:09.114619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a278b-8a1d-4b3e-a324-f8484c2ad28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:09.115805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a278b-8a1d-4b3e-a324-f8484c2ad28d None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:09.116540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a278b-8a1d-4b3e-a324-f8484c2ad28d None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:09.117009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a278b-8a1d-4b3e-a324-f8484c2ad28d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:09.129636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a278b-8a1d-4b3e-a324-f8484c2ad28d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-3f2a278b-8a1d-4b3e-a324-f8484c2ad28d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:09.135230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:09.136491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:09.137289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:09.139910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 104/200] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:09.148089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52370184-1e86-4e7f-b0a4-0ed4215f2da2 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:09.261647 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52370184-1e86-4e7f-b0a4-0ed4215f2da2 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:09.297850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738384b-4ee7-4494-b60b-a6adfdf92776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:09.298461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738384b-4ee7-4494-b60b-a6adfdf92776 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:09.298743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738384b-4ee7-4494-b60b-a6adfdf92776 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:09.299010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738384b-4ee7-4494-b60b-a6adfdf92776 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:09.299675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738384b-4ee7-4494-b60b-a6adfdf92776 None admin] RBAC: Adding query filter params (name=member) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:09.300007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738384b-4ee7-4494-b60b-a6adfdf92776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:09.301381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738384b-4ee7-4494-b60b-a6adfdf92776 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:09.315124 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 97/201] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:09.316631 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a278b-8a1d-4b3e-a324-f8484c2ad28d None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:09.423345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:09.435638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a278b-8a1d-4b3e-a324-f8484c2ad28d None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:09.472140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b18c32-7ed2-461f-89fa-f7bf0b2713ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:09.472784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b18c32-7ed2-461f-89fa-f7bf0b2713ff None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:09.473062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b18c32-7ed2-461f-89fa-f7bf0b2713ff None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:09.473335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b18c32-7ed2-461f-89fa-f7bf0b2713ff None admin] PATH_INFO: `/v3/users/88fc56cff73e41bfa79becb4c3b65f21` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:09.474493 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b18c32-7ed2-461f-89fa-f7bf0b2713ff None admin] RBAC: Authorizing `identity:get_user(user_id=88fc56cff73e41bfa79becb4c3b65f21)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:09.476911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b18c32-7ed2-461f-89fa-f7bf0b2713ff None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:09.479424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 105/202] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:09 2022] GET /identity/v3/users/88fc56cff73e41bfa79becb4c3b65f21 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:09.486034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738384b-4ee7-4494-b60b-a6adfdf92776 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:09.511214 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:09.618593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738384b-4ee7-4494-b60b-a6adfdf92776 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:09.657016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14e865-4175-43c8-b3f7-77f93cab8628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:09.657646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14e865-4175-43c8-b3f7-77f93cab8628 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:09.657923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14e865-4175-43c8-b3f7-77f93cab8628 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:09.658193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14e865-4175-43c8-b3f7-77f93cab8628 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:09.734888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14e865-4175-43c8-b3f7-77f93cab8628 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:09.736317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14e865-4175-43c8-b3f7-77f93cab8628 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:09.755504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:09.841550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 98/203] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:09.847292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b18c32-7ed2-461f-89fa-f7bf0b2713ff None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:09.955524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b18c32-7ed2-461f-89fa-f7bf0b2713ff None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:09.991344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18534c0-02b4-4cbb-bb29-216e888212d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:09.991869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18534c0-02b4-4cbb-bb29-216e888212d6 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:09.992172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18534c0-02b4-4cbb-bb29-216e888212d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:09.992442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18534c0-02b4-4cbb-bb29-216e888212d6 None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:09.993573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18534c0-02b4-4cbb-bb29-216e888212d6 None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:09.995024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18534c0-02b4-4cbb-bb29-216e888212d6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:10.001544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 106/204] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:09 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:10.002887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14e865-4175-43c8-b3f7-77f93cab8628 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:10.117505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14e865-4175-43c8-b3f7-77f93cab8628 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:10.135371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:10.153519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21903fec-b340-465e-a1b7-7038b45005d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:10.154121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21903fec-b340-465e-a1b7-7038b45005d3 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:10.154396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21903fec-b340-465e-a1b7-7038b45005d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:10.154665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21903fec-b340-465e-a1b7-7038b45005d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:10.155391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21903fec-b340-465e-a1b7-7038b45005d3 None admin] RBAC: Adding query filter params (role.id=9ab8d70da5894a2f99c6d22e5351baa3, scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5, user.id=88fc56cff73e41bfa79becb4c3b65f21) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:10.155699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21903fec-b340-465e-a1b7-7038b45005d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:10.157161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21903fec-b340-465e-a1b7-7038b45005d3 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:10.163203 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 99/205] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:42:10 2022] GET /identity/v3/role_assignments?user.id=88fc56cff73e41bfa79becb4c3b65f21&scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5&role.id=9ab8d70da5894a2f99c6d22e5351baa3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:10.964672 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f9543-e655-40c3-9a3c-560949ce16fe None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:10.965245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f9543-e655-40c3-9a3c-560949ce16fe None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:10.965245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f9543-e655-40c3-9a3c-560949ce16fe None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:10.966696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 107/206] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:10.971731 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df0276-9553-418b-a87b-dca202915cd4 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:10.972056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df0276-9553-418b-a87b-dca202915cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:10.972334 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df0276-9553-418b-a87b-dca202915cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:10.991865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-32df0276-9553-418b-a87b-dca202915cd4 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:10.993102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-32df0276-9553-418b-a87b-dca202915cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:11.030621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a065f63-a584-4b0b-b576-b1c57e60004d None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:11.030933 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a065f63-a584-4b0b-b576-b1c57e60004d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:11.031166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a065f63-a584-4b0b-b576-b1c57e60004d None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:11.032537 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 108/207] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:42:11.037658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63b51-551e-466d-b52e-654e5b58545e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:11.037938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63b51-551e-466d-b52e-654e5b58545e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:11.038207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63b51-551e-466d-b52e-654e5b58545e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:11.055868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7db63b51-551e-466d-b52e-654e5b58545e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:11.057068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7db63b51-551e-466d-b52e-654e5b58545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:11.065562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 100/208] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:11.077882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:11.126171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 109/209] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:11.339350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df0276-9553-418b-a87b-dca202915cd4 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:11.355536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c9258-383c-44ab-b315-3e08d78e2c2b None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:11.355822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c9258-383c-44ab-b315-3e08d78e2c2b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:11.356123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c9258-383c-44ab-b315-3e08d78e2c2b None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:11.357604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 110/210] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:42:11.362457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae754351-c563-4262-8013-57b665554c1d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:11.362746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae754351-c563-4262-8013-57b665554c1d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:11.363017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae754351-c563-4262-8013-57b665554c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:11.382738 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ae754351-c563-4262-8013-57b665554c1d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:11.384372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ae754351-c563-4262-8013-57b665554c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:11.444348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df0276-9553-418b-a87b-dca202915cd4 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:11.454565 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 111/211] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:11.455399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae754351-c563-4262-8013-57b665554c1d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:11.458026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:11.485852 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e030cc-9049-4ce5-8b00-2c0aec841e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:11.486583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e030cc-9049-4ce5-8b00-2c0aec841e80 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:11.486853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e030cc-9049-4ce5-8b00-2c0aec841e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:11.487117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e030cc-9049-4ce5-8b00-2c0aec841e80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:11.497451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e030cc-9049-4ce5-8b00-2c0aec841e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-56e030cc-9049-4ce5-8b00-2c0aec841e80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:11.502549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:11.503688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:11.504534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 101/212] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:11.511693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e030cc-9049-4ce5-8b00-2c0aec841e80 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:11.544867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:11.562668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae754351-c563-4262-8013-57b665554c1d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:11.598123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca2752-6ac2-4135-a00c-138c3f36fca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:11.598669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2752-6ac2-4135-a00c-138c3f36fca1 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:11.598939 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2752-6ac2-4135-a00c-138c3f36fca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:11.599202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2752-6ac2-4135-a00c-138c3f36fca1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:11.600396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca2752-6ac2-4135-a00c-138c3f36fca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:11.601970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca2752-6ac2-4135-a00c-138c3f36fca1 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:11.603889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 112/213] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:11.608765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca2752-6ac2-4135-a00c-138c3f36fca1 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:11.624828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e030cc-9049-4ce5-8b00-2c0aec841e80 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:11.662888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d25318-3d4f-4f7f-b2ba-57731c64c14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:11.663588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d25318-3d4f-4f7f-b2ba-57731c64c14c None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:11.663862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d25318-3d4f-4f7f-b2ba-57731c64c14c None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:11.664156 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d25318-3d4f-4f7f-b2ba-57731c64c14c None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:11.664808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d25318-3d4f-4f7f-b2ba-57731c64c14c None admin] RBAC: Adding query filter params (name=member) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:11.665104 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d25318-3d4f-4f7f-b2ba-57731c64c14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:11.666485 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d25318-3d4f-4f7f-b2ba-57731c64c14c None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:11.674740 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 102/214] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:11.680904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d25318-3d4f-4f7f-b2ba-57731c64c14c None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:11.709458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca2752-6ac2-4135-a00c-138c3f36fca1 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:11.717198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:11.744720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9757d-03b4-4512-af31-a160cd214d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:11.745322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9757d-03b4-4512-af31-a160cd214d0a None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:11.745600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9757d-03b4-4512-af31-a160cd214d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:11.745869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9757d-03b4-4512-af31-a160cd214d0a None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:11.746513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9757d-03b4-4512-af31-a160cd214d0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:11.747989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9757d-03b4-4512-af31-a160cd214d0a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:11.770883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 113/215] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:42:11 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:42:11.771713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9757d-03b4-4512-af31-a160cd214d0a None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:11.789441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:11.796152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d25318-3d4f-4f7f-b2ba-57731c64c14c None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:11.840860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec92d1-bed6-47de-95f5-990eb2863bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:11.841893 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec92d1-bed6-47de-95f5-990eb2863bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:11.842279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec92d1-bed6-47de-95f5-990eb2863bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:11.842450 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec92d1-bed6-47de-95f5-990eb2863bb9 None admin] PATH_INFO: `/v3/users/dd2ef095304a45f9864491bcc9ff77da` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:11.858283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec92d1-bed6-47de-95f5-990eb2863bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=dd2ef095304a45f9864491bcc9ff77da)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:11.859699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec92d1-bed6-47de-95f5-990eb2863bb9 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:11.861678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 103/216] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:11 2022] GET /identity/v3/users/dd2ef095304a45f9864491bcc9ff77da => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:11.868099 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec92d1-bed6-47de-95f5-990eb2863bb9 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:11.894679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9757d-03b4-4512-af31-a160cd214d0a None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:11.935457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a1fed-0179-45e3-a2b9-d3e8221cd481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:11.936183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a1fed-0179-45e3-a2b9-d3e8221cd481 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:11.936464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a1fed-0179-45e3-a2b9-d3e8221cd481 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:11.936731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a1fed-0179-45e3-a2b9-d3e8221cd481 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:11.948622 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a1fed-0179-45e3-a2b9-d3e8221cd481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-7a6a1fed-0179-45e3-a2b9-d3e8221cd481 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:11.954019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:11.955826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:11.956668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 114/217] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:11.961418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a1fed-0179-45e3-a2b9-d3e8221cd481 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:11.979756 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:11.993111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec92d1-bed6-47de-95f5-990eb2863bb9 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:12.034969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cc83d-1f1a-4368-9ed4-3e63caf1ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:12.035811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cc83d-1f1a-4368-9ed4-3e63caf1ab92 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:12.036111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cc83d-1f1a-4368-9ed4-3e63caf1ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:12.036378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cc83d-1f1a-4368-9ed4-3e63caf1ab92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:12.037098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2cc83d-1f1a-4368-9ed4-3e63caf1ab92 None admin] RBAC: Adding query filter params (role.id=9ab8d70da5894a2f99c6d22e5351baa3, user.id=dd2ef095304a45f9864491bcc9ff77da, scope.system=all) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:12.037397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2cc83d-1f1a-4368-9ed4-3e63caf1ab92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:12.041970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2cc83d-1f1a-4368-9ed4-3e63caf1ab92 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:12.055722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 104/218] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Oct 14 17:42:11 2022] GET /identity/v3/role_assignments?user.id=dd2ef095304a45f9864491bcc9ff77da&scope.system=all&role.id=9ab8d70da5894a2f99c6d22e5351baa3 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:12.065256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:12.106489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a1fed-0179-45e3-a2b9-d3e8221cd481 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:12.148547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b3566-d18c-4d3b-9ac0-07b069eb8b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:12.149102 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b3566-d18c-4d3b-9ac0-07b069eb8b58 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:12.149379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b3566-d18c-4d3b-9ac0-07b069eb8b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:12.149652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b3566-d18c-4d3b-9ac0-07b069eb8b58 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:12.150497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b3566-d18c-4d3b-9ac0-07b069eb8b58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:12.150806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b3566-d18c-4d3b-9ac0-07b069eb8b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:12.152790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b3566-d18c-4d3b-9ac0-07b069eb8b58 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:12.162516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 115/219] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:12.168672 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cc83d-1f1a-4368-9ed4-3e63caf1ab92 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:12.293893 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cc83d-1f1a-4368-9ed4-3e63caf1ab92 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:12.329904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806d54f-7a95-405a-87da-306480bf840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:12.330532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806d54f-7a95-405a-87da-306480bf840b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:12.330804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806d54f-7a95-405a-87da-306480bf840b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:12.331073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806d54f-7a95-405a-87da-306480bf840b None admin] PATH_INFO: `/v3/users/5dbd9560ac48432a903adec3848c9638` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:12.332227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806d54f-7a95-405a-87da-306480bf840b None admin] RBAC: Authorizing `identity:get_user(user_id=5dbd9560ac48432a903adec3848c9638)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:12.333719 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806d54f-7a95-405a-87da-306480bf840b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:12.336365 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 105/220] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:12 2022] GET /identity/v3/users/5dbd9560ac48432a903adec3848c9638 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:12.341468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b3566-d18c-4d3b-9ac0-07b069eb8b58 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:12.460715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b3566-d18c-4d3b-9ac0-07b069eb8b58 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:12.496432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:12.501884 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a0ee1-f82c-443c-98d4-9b4285270e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:12.502500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a0ee1-f82c-443c-98d4-9b4285270e80 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:12.502774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a0ee1-f82c-443c-98d4-9b4285270e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:12.503039 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a0ee1-f82c-443c-98d4-9b4285270e80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:12.576636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a0ee1-f82c-443c-98d4-9b4285270e80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:12.578111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a0ee1-f82c-443c-98d4-9b4285270e80 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:12.686313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 116/221] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:12.691759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806d54f-7a95-405a-87da-306480bf840b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:12.719994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:12.801371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806d54f-7a95-405a-87da-306480bf840b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:12.839591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5c39b-5ac4-4048-bb01-a631eeb8eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:12.840628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5c39b-5ac4-4048-bb01-a631eeb8eda8 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:12.841106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5c39b-5ac4-4048-bb01-a631eeb8eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:12.841606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5c39b-5ac4-4048-bb01-a631eeb8eda8 None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:12.843072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5c39b-5ac4-4048-bb01-a631eeb8eda8 None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:12.844910 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5c39b-5ac4-4048-bb01-a631eeb8eda8 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:12.846841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 106/222] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:12 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:12.859916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a0ee1-f82c-443c-98d4-9b4285270e80 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:12.887595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:12.967569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a0ee1-f82c-443c-98d4-9b4285270e80 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:13.004229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cf25e-4208-4c76-a4dc-1c610e32a3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:13.004749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cf25e-4208-4c76-a4dc-1c610e32a3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:13.005019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cf25e-4208-4c76-a4dc-1c610e32a3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:13.005283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cf25e-4208-4c76-a4dc-1c610e32a3d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:13.005996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cf25e-4208-4c76-a4dc-1c610e32a3d6 None admin] RBAC: Adding query filter params (role.id=6d101684527d45fabea90eeb60fc3e99, scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5, user.id=5dbd9560ac48432a903adec3848c9638) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:13.007371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cf25e-4208-4c76-a4dc-1c610e32a3d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:13.008955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cf25e-4208-4c76-a4dc-1c610e32a3d6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:13.015660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 117/223] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:42:12 2022] GET /identity/v3/role_assignments?user.id=5dbd9560ac48432a903adec3848c9638&scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5&role.id=6d101684527d45fabea90eeb60fc3e99 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:14.142746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c7e2d-1564-420d-b15c-fd6226c29d9d None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.143302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c7e2d-1564-420d-b15c-fd6226c29d9d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.143302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c7e2d-1564-420d-b15c-fd6226c29d9d None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.144767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 107/224] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:14.150444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fc76c-cef8-4f98-894d-3ac6fbfd75bf None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.150874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fc76c-cef8-4f98-894d-3ac6fbfd75bf None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.151044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fc76c-cef8-4f98-894d-3ac6fbfd75bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.169693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-741fc76c-cef8-4f98-894d-3ac6fbfd75bf None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:14.170870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-741fc76c-cef8-4f98-894d-3ac6fbfd75bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:14.229846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff63bcf-066a-4a4e-b41a-7349d1c6fa8c None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.230268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff63bcf-066a-4a4e-b41a-7349d1c6fa8c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.230479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff63bcf-066a-4a4e-b41a-7349d1c6fa8c None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.231795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 108/225] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:42:14.237906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6800a-e8d4-46ee-a203-4a48b6feb66a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.238186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6800a-e8d4-46ee-a203-4a48b6feb66a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.238452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6800a-e8d4-46ee-a203-4a48b6feb66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.246917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 118/226] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:14.257622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-bae6800a-e8d4-46ee-a203-4a48b6feb66a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:14.258797 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-bae6800a-e8d4-46ee-a203-4a48b6feb66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:14.320594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8043a-ecd2-4f01-941a-807846605e26 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.320915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8043a-ecd2-4f01-941a-807846605e26 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.321135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8043a-ecd2-4f01-941a-807846605e26 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.322594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 119/227] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:42:14.325747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 109/228] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:14.329818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b116dc-755f-4b82-b070-a2df6854b495 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.330163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b116dc-755f-4b82-b070-a2df6854b495 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.330497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b116dc-755f-4b82-b070-a2df6854b495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.351142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-42b116dc-755f-4b82-b070-a2df6854b495 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:14.352354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-42b116dc-755f-4b82-b070-a2df6854b495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:14.373838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:14.424251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 120/229] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:14.525301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6800a-e8d4-46ee-a203-4a48b6feb66a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:14.550465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:14.595820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b116dc-755f-4b82-b070-a2df6854b495 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:14.630084 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6800a-e8d4-46ee-a203-4a48b6feb66a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:14.667290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d658d-f3c8-4444-8f48-5677b5a58d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:14.668106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d658d-f3c8-4444-8f48-5677b5a58d23 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.668387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d658d-f3c8-4444-8f48-5677b5a58d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.669423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d658d-f3c8-4444-8f48-5677b5a58d23 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.681150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d658d-f3c8-4444-8f48-5677b5a58d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-b45d658d-f3c8-4444-8f48-5677b5a58d23 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:14.686935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:14.687816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:14.688908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 110/230] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:14.695821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d658d-f3c8-4444-8f48-5677b5a58d23 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:14.701063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b116dc-755f-4b82-b070-a2df6854b495 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:14.734996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe5451-7d98-4e0a-b1ce-5d1b38cf72ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:14.735621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe5451-7d98-4e0a-b1ce-5d1b38cf72ac None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.735916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe5451-7d98-4e0a-b1ce-5d1b38cf72ac None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.736190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe5451-7d98-4e0a-b1ce-5d1b38cf72ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.737276 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe5451-7d98-4e0a-b1ce-5d1b38cf72ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:14.738719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe5451-7d98-4e0a-b1ce-5d1b38cf72ac None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:14.740313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 121/231] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:14.740313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe5451-7d98-4e0a-b1ce-5d1b38cf72ac None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:14.766803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:14.779414 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:14.797773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d658d-f3c8-4444-8f48-5677b5a58d23 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:14.831098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ceeb0-f3cc-4114-8fdb-466b89bc9cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:14.831709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ceeb0-f3cc-4114-8fdb-466b89bc9cbd None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.832007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ceeb0-f3cc-4114-8fdb-466b89bc9cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.832272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ceeb0-f3cc-4114-8fdb-466b89bc9cbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.841906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe5451-7d98-4e0a-b1ce-5d1b38cf72ac None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:14.843137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ceeb0-f3cc-4114-8fdb-466b89bc9cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-b70ceeb0-f3cc-4114-8fdb-466b89bc9cbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:14.847924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:14.848807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:14.849609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 111/232] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:14.850949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ceeb0-f3cc-4114-8fdb-466b89bc9cbd None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:14.877762 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f6175-fa40-4baf-a6a4-18982ae43cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:14.878445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f6175-fa40-4baf-a6a4-18982ae43cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.878791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f6175-fa40-4baf-a6a4-18982ae43cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.879130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f6175-fa40-4baf-a6a4-18982ae43cc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.879863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f6175-fa40-4baf-a6a4-18982ae43cc3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:14.880198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f6175-fa40-4baf-a6a4-18982ae43cc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:14.882233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f6175-fa40-4baf-a6a4-18982ae43cc3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:14.890402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 122/233] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:14.890402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f6175-fa40-4baf-a6a4-18982ae43cc3 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:14.951563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ceeb0-f3cc-4114-8fdb-466b89bc9cbd None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:14.974433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:14.985317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b799b8-9d0b-4aca-8780-c84ff0414441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:14.985920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b799b8-9d0b-4aca-8780-c84ff0414441 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:14.986191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b799b8-9d0b-4aca-8780-c84ff0414441 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:14.986490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b799b8-9d0b-4aca-8780-c84ff0414441 None admin] PATH_INFO: `/v3/groups` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:14.987107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b799b8-9d0b-4aca-8780-c84ff0414441 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:14.988575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b799b8-9d0b-4aca-8780-c84ff0414441 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:14.989845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f6175-fa40-4baf-a6a4-18982ae43cc3 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:15.003979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 112/234] 10.222.0.27 () {60 vars in 1134 bytes} [Fri Oct 14 17:42:14 2022] POST /identity/v3/groups => generated 229 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:42:15.004606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b799b8-9d0b-4aca-8780-c84ff0414441 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:15.013198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:15.032118 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab438c8-a5a8-491b-aed4-3937cf7eccda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:15.032631 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab438c8-a5a8-491b-aed4-3937cf7eccda None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:15.032903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab438c8-a5a8-491b-aed4-3937cf7eccda None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:15.033177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab438c8-a5a8-491b-aed4-3937cf7eccda None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:15.033863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab438c8-a5a8-491b-aed4-3937cf7eccda None admin] RBAC: Adding query filter params (name=member) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:15.034161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab438c8-a5a8-491b-aed4-3937cf7eccda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:15.035698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab438c8-a5a8-491b-aed4-3937cf7eccda None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:15.047967 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 123/235] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:15.052225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab438c8-a5a8-491b-aed4-3937cf7eccda None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:15.140200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b799b8-9d0b-4aca-8780-c84ff0414441 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:15.167140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab438c8-a5a8-491b-aed4-3937cf7eccda None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:15.181375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74166996-2211-40e6-8199-7344cea450ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:15.182078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74166996-2211-40e6-8199-7344cea450ee None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:15.182278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74166996-2211-40e6-8199-7344cea450ee None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:15.182544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74166996-2211-40e6-8199-7344cea450ee None admin] PATH_INFO: `/v3/users/0e2a1a82c5c84eb7a4b073838d953bab` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:15.198573 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74166996-2211-40e6-8199-7344cea450ee None admin] RBAC: Authorizing `identity:get_user(user_id=0e2a1a82c5c84eb7a4b073838d953bab)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:15.200077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74166996-2211-40e6-8199-7344cea450ee None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:15.201784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 113/236] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:15 2022] GET /identity/v3/users/0e2a1a82c5c84eb7a4b073838d953bab => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:15.204770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187d5bd-5ec9-4f65-80cf-e24dba5a0a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:15.205396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187d5bd-5ec9-4f65-80cf-e24dba5a0a39 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:15.205674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187d5bd-5ec9-4f65-80cf-e24dba5a0a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:15.205934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187d5bd-5ec9-4f65-80cf-e24dba5a0a39 None admin] PATH_INFO: `/v3/users/dd2ef095304a45f9864491bcc9ff77da` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:15.207058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187d5bd-5ec9-4f65-80cf-e24dba5a0a39 None admin] RBAC: Authorizing `identity:get_user(user_id=dd2ef095304a45f9864491bcc9ff77da)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:15.207733 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74166996-2211-40e6-8199-7344cea450ee None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:15.208538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187d5bd-5ec9-4f65-80cf-e24dba5a0a39 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:15.210206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 124/237] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:15 2022] GET /identity/v3/users/dd2ef095304a45f9864491bcc9ff77da => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:15.215719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187d5bd-5ec9-4f65-80cf-e24dba5a0a39 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:15.236385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:15.278435 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:15.322263 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74166996-2211-40e6-8199-7344cea450ee None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:15.327455 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187d5bd-5ec9-4f65-80cf-e24dba5a0a39 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:15.363652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe5c06-9026-4a33-be38-cb14dcc68390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:15.364245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb36f9-099a-4c02-82d1-68325173a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:15.364362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe5c06-9026-4a33-be38-cb14dcc68390 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:15.364545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe5c06-9026-4a33-be38-cb14dcc68390 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:15.364815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe5c06-9026-4a33-be38-cb14dcc68390 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:15.364971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb36f9-099a-4c02-82d1-68325173a12d None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:15.365230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb36f9-099a-4c02-82d1-68325173a12d None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:15.365506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb36f9-099a-4c02-82d1-68325173a12d None admin] PATH_INFO: `/v3/system/users/dd2ef095304a45f9864491bcc9ff77da/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:15.365632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe5c06-9026-4a33-be38-cb14dcc68390 None admin] RBAC: Adding query filter params (role.id=6d101684527d45fabea90eeb60fc3e99, user.id=0e2a1a82c5c84eb7a4b073838d953bab, scope.system=all) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:15.366903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe5c06-9026-4a33-be38-cb14dcc68390 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:15.367525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb36f9-099a-4c02-82d1-68325173a12d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dd2ef095304a45f9864491bcc9ff77da, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:15.368275 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe5c06-9026-4a33-be38-cb14dcc68390 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:15.368958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb36f9-099a-4c02-82d1-68325173a12d None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:15.373828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 114/238] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Oct 14 17:42:15 2022] GET /identity/v3/role_assignments?user.id=0e2a1a82c5c84eb7a4b073838d953bab&scope.system=all&role.id=6d101684527d45fabea90eeb60fc3e99 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:15.396272 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 125/239] 10.222.0.27 () {58 vars in 1334 bytes} [Fri Oct 14 17:42:15 2022] PUT /identity/v3/system/users/dd2ef095304a45f9864491bcc9ff77da/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:42:17.218404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daafad-37ce-486c-be7a-571ffb40dc19 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:17.218926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daafad-37ce-486c-be7a-571ffb40dc19 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:17.218926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daafad-37ce-486c-be7a-571ffb40dc19 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:17.220443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 115/240] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:17.225447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c83fb8-1eda-4faa-9d57-e588a0515f48 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:17.225829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c83fb8-1eda-4faa-9d57-e588a0515f48 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:17.225984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c83fb8-1eda-4faa-9d57-e588a0515f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:17.244415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c8c83fb8-1eda-4faa-9d57-e588a0515f48 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:17.245589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c8c83fb8-1eda-4faa-9d57-e588a0515f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:17.313959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 126/241] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:17.563764 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167ad78-0be1-463b-8dfb-a72ccd730d01 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:17.564185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167ad78-0be1-463b-8dfb-a72ccd730d01 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:17.564381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167ad78-0be1-463b-8dfb-a72ccd730d01 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:17.565694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 116/242] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:17.571011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b359c44-f91c-45b4-a1c4-d638a666c9bf None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:17.571295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b359c44-f91c-45b4-a1c4-d638a666c9bf None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:17.571557 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b359c44-f91c-45b4-a1c4-d638a666c9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:17.571904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f13726-4c97-4355-a792-34788651ce20 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:17.572181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f13726-4c97-4355-a792-34788651ce20 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:17.572456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f13726-4c97-4355-a792-34788651ce20 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:17.573726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 117/243] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:17.578480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e2ccc-f96e-4d70-8e1e-2eb8b8c49046 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:17.579644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e2ccc-f96e-4d70-8e1e-2eb8b8c49046 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:17.579644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e2ccc-f96e-4d70-8e1e-2eb8b8c49046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:17.590119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4b359c44-f91c-45b4-a1c4-d638a666c9bf None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:17.591295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4b359c44-f91c-45b4-a1c4-d638a666c9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:17.594085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:17.596799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-3a5e2ccc-f96e-4d70-8e1e-2eb8b8c49046 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:17.597954 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3a5e2ccc-f96e-4d70-8e1e-2eb8b8c49046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:17.659289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 127/244] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:17.664316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b359c44-f91c-45b4-a1c4-d638a666c9bf None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:17.664666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 118/245] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:17.763742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b359c44-f91c-45b4-a1c4-d638a666c9bf None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:17.797620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fa130-d8fe-4aa4-abf8-408614d2e6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:17.798242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fa130-d8fe-4aa4-abf8-408614d2e6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:17.798517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fa130-d8fe-4aa4-abf8-408614d2e6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:17.798843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fa130-d8fe-4aa4-abf8-408614d2e6d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:17.799955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2fa130-d8fe-4aa4-abf8-408614d2e6d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:17.801406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2fa130-d8fe-4aa4-abf8-408614d2e6d1 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:17.803105 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 128/246] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:17.808634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e2ccc-f96e-4d70-8e1e-2eb8b8c49046 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:17.866060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:17.912089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e2ccc-f96e-4d70-8e1e-2eb8b8c49046 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:17.926585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fa130-d8fe-4aa4-abf8-408614d2e6d1 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:17.937773 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:17.949736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86918bf7-6f4c-4679-9c55-2c2be5b0115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:17.950351 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86918bf7-6f4c-4679-9c55-2c2be5b0115c None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:17.950624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86918bf7-6f4c-4679-9c55-2c2be5b0115c None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:17.950892 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86918bf7-6f4c-4679-9c55-2c2be5b0115c None admin] PATH_INFO: `/v3/groups` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:17.951513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86918bf7-6f4c-4679-9c55-2c2be5b0115c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:17.953078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86918bf7-6f4c-4679-9c55-2c2be5b0115c None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:17.968613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 119/247] 10.222.0.27 () {60 vars in 1134 bytes} [Fri Oct 14 17:42:17 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:42:17.972949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86918bf7-6f4c-4679-9c55-2c2be5b0115c None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:18.045398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fa130-d8fe-4aa4-abf8-408614d2e6d1 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:18.089961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97e135-0072-4d21-9d66-7e45dca16b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:18.090602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97e135-0072-4d21-9d66-7e45dca16b96 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:18.090884 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97e135-0072-4d21-9d66-7e45dca16b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:18.091133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97e135-0072-4d21-9d66-7e45dca16b96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:18.092758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86918bf7-6f4c-4679-9c55-2c2be5b0115c None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:18.103120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97e135-0072-4d21-9d66-7e45dca16b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-ee97e135-0072-4d21-9d66-7e45dca16b96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:18.108230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:18.109187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:18.109990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 129/248] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:18.111863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:18.115475 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97e135-0072-4d21-9d66-7e45dca16b96 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:18.136174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b330e7c-ab86-47ba-9cc3-66e0f0b208b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:18.136980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b330e7c-ab86-47ba-9cc3-66e0f0b208b1 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:18.137266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b330e7c-ab86-47ba-9cc3-66e0f0b208b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:18.137537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b330e7c-ab86-47ba-9cc3-66e0f0b208b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:18.150227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b330e7c-ab86-47ba-9cc3-66e0f0b208b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-0b330e7c-ab86-47ba-9cc3-66e0f0b208b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:18.155262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:18.156232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:18.157554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 120/249] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:18.162129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b330e7c-ab86-47ba-9cc3-66e0f0b208b1 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:18.195612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:18.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97e135-0072-4d21-9d66-7e45dca16b96 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:18.280576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc4045-ba10-425a-9594-f14801a8c276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:18.281192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc4045-ba10-425a-9594-f14801a8c276 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:18.281462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc4045-ba10-425a-9594-f14801a8c276 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:18.281785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc4045-ba10-425a-9594-f14801a8c276 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:18.282451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc4045-ba10-425a-9594-f14801a8c276 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:18.282748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc4045-ba10-425a-9594-f14801a8c276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:18.282915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b330e7c-ab86-47ba-9cc3-66e0f0b208b1 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:18.284185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc4045-ba10-425a-9594-f14801a8c276 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:18.294994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 130/250] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:18.300716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc4045-ba10-425a-9594-f14801a8c276 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:18.335324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01497253-243c-4204-94c1-6aa6468a38e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:18.336234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497253-243c-4204-94c1-6aa6468a38e4 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:18.336584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497253-243c-4204-94c1-6aa6468a38e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:18.336872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497253-243c-4204-94c1-6aa6468a38e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:18.337639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01497253-243c-4204-94c1-6aa6468a38e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:18.338030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01497253-243c-4204-94c1-6aa6468a38e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:18.339521 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01497253-243c-4204-94c1-6aa6468a38e4 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:18.349169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 121/251] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:18.355484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01497253-243c-4204-94c1-6aa6468a38e4 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:18.408436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:18.436885 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc4045-ba10-425a-9594-f14801a8c276 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:18.477569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:18.483607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efcd48-f7d3-4eb6-8b83-ebbb3d32f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:18.484343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efcd48-f7d3-4eb6-8b83-ebbb3d32f618 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:18.485232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efcd48-f7d3-4eb6-8b83-ebbb3d32f618 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:18.485536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efcd48-f7d3-4eb6-8b83-ebbb3d32f618 None admin] PATH_INFO: `/v3/users/0e2a1a82c5c84eb7a4b073838d953bab` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:18.487002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efcd48-f7d3-4eb6-8b83-ebbb3d32f618 None admin] RBAC: Authorizing `identity:get_user(user_id=0e2a1a82c5c84eb7a4b073838d953bab)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:18.488707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efcd48-f7d3-4eb6-8b83-ebbb3d32f618 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:18.491393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 131/252] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:18 2022] GET /identity/v3/users/0e2a1a82c5c84eb7a4b073838d953bab => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:18.497615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efcd48-f7d3-4eb6-8b83-ebbb3d32f618 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:18.499426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01497253-243c-4204-94c1-6aa6468a38e4 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:18.541575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07049fa3-b7a9-44e0-a14e-1f5da3e9aa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:18.542544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07049fa3-b7a9-44e0-a14e-1f5da3e9aa05 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:18.543011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07049fa3-b7a9-44e0-a14e-1f5da3e9aa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:18.543487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07049fa3-b7a9-44e0-a14e-1f5da3e9aa05 None admin] PATH_INFO: `/v3/users/dd2ef095304a45f9864491bcc9ff77da` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:18.545277 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07049fa3-b7a9-44e0-a14e-1f5da3e9aa05 None admin] RBAC: Authorizing `identity:get_user(user_id=dd2ef095304a45f9864491bcc9ff77da)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:18.546922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07049fa3-b7a9-44e0-a14e-1f5da3e9aa05 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:18.548845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 122/253] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:18 2022] GET /identity/v3/users/dd2ef095304a45f9864491bcc9ff77da => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:18.555178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07049fa3-b7a9-44e0-a14e-1f5da3e9aa05 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:18.641717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efcd48-f7d3-4eb6-8b83-ebbb3d32f618 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:18.692802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07049fa3-b7a9-44e0-a14e-1f5da3e9aa05 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:18.697266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:18.710441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e061d-3c5c-481b-8dd2-01fb59862927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:18.711093 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e061d-3c5c-481b-8dd2-01fb59862927 None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:18.711370 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e061d-3c5c-481b-8dd2-01fb59862927 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:18.711641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e061d-3c5c-481b-8dd2-01fb59862927 None admin] PATH_INFO: `/v3/system/users/0e2a1a82c5c84eb7a4b073838d953bab/roles/6d101684527d45fabea90eeb60fc3e99` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:18.715667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e061d-3c5c-481b-8dd2-01fb59862927 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0e2a1a82c5c84eb7a4b073838d953bab, role_id=6d101684527d45fabea90eeb60fc3e99)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:18.717195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e061d-3c5c-481b-8dd2-01fb59862927 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:18.726399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 132/254] 10.222.0.27 () {58 vars in 1334 bytes} [Fri Oct 14 17:42:18 2022] PUT /identity/v3/system/users/0e2a1a82c5c84eb7a4b073838d953bab/roles/6d101684527d45fabea90eeb60fc3e99 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:42:18.747407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06983f2f-9211-4dcb-b731-8719ac60529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:18.748448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06983f2f-9211-4dcb-b731-8719ac60529c None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:18.748973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06983f2f-9211-4dcb-b731-8719ac60529c None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:18.749439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06983f2f-9211-4dcb-b731-8719ac60529c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:18.750389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06983f2f-9211-4dcb-b731-8719ac60529c None admin] RBAC: Adding query filter params (role.id=9ab8d70da5894a2f99c6d22e5351baa3, user.id=dd2ef095304a45f9864491bcc9ff77da, scope.system=all) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:18.750936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06983f2f-9211-4dcb-b731-8719ac60529c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:18.752823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06983f2f-9211-4dcb-b731-8719ac60529c None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:18.768769 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 123/255] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Oct 14 17:42:18 2022] GET /identity/v3/role_assignments?user.id=dd2ef095304a45f9864491bcc9ff77da&scope.system=all&role.id=9ab8d70da5894a2f99c6d22e5351baa3 => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:20.449361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eae5f3-7e0e-4559-870a-d45c2549e781 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:20.450226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eae5f3-7e0e-4559-870a-d45c2549e781 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:20.450663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eae5f3-7e0e-4559-870a-d45c2549e781 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:20.452463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 133/256] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:20.469692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df256bec-c3c5-444e-9197-33cd63d36740 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:20.470592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df256bec-c3c5-444e-9197-33cd63d36740 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:20.471030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df256bec-c3c5-444e-9197-33cd63d36740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:20.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-df256bec-c3c5-444e-9197-33cd63d36740 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:20.498291 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-df256bec-c3c5-444e-9197-33cd63d36740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:20.578082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 124/257] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 17:42:20.905033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e43c92-ef12-4c67-a904-f894011dabff None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:20.905726 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e43c92-ef12-4c67-a904-f894011dabff None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:20.906196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e43c92-ef12-4c67-a904-f894011dabff None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:20.908000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 134/258] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:20.915564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617f726-0951-44ff-8d80-fee93ef5eb0a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:20.916100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617f726-0951-44ff-8d80-fee93ef5eb0a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:20.916584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617f726-0951-44ff-8d80-fee93ef5eb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:20.946437 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1617f726-0951-44ff-8d80-fee93ef5eb0a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:20.947999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1617f726-0951-44ff-8d80-fee93ef5eb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:20.958937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 125/259] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 17:42:21.174700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e43c92-ef12-4c67-a904-f894011dabff None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:21.191641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e43c92-ef12-4c67-a904-f894011dabff None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:21.197485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83521b8c-ec9f-4d26-969d-3cf972187313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:21.198358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83521b8c-ec9f-4d26-969d-3cf972187313 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.198821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83521b8c-ec9f-4d26-969d-3cf972187313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.199313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83521b8c-ec9f-4d26-969d-3cf972187313 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.217405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83521b8c-ec9f-4d26-969d-3cf972187313 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-83521b8c-ec9f-4d26-969d-3cf972187313 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:21.222731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:21.223676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:21.224489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:21.231999 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 135/260] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Oct 14 17:42:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 54 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:21.234662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617f726-0951-44ff-8d80-fee93ef5eb0a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:21.246183 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617f726-0951-44ff-8d80-fee93ef5eb0a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:21.251988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf1d16-8788-4d01-b87a-9b08e27f0ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:21.252811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf1d16-8788-4d01-b87a-9b08e27f0ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.253275 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf1d16-8788-4d01-b87a-9b08e27f0ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.253751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf1d16-8788-4d01-b87a-9b08e27f0ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.254619 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf1d16-8788-4d01-b87a-9b08e27f0ce3 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:21.255116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf1d16-8788-4d01-b87a-9b08e27f0ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:21.257771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: WARNING py.warnings [None req-c5cf1d16-8788-4d01-b87a-9b08e27f0ce3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:42:21.257771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: warnings.warn(msg) Oct 14 17:42:21.257771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: Oct 14 17:42:21.259026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf1d16-8788-4d01-b87a-9b08e27f0ce3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:21.271821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 126/261] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:42:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:21.278607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83521b8c-ec9f-4d26-969d-3cf972187313 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:21.293483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83521b8c-ec9f-4d26-969d-3cf972187313 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:21.298156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a4c24-19d9-41d4-8c5d-525aee09713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:21.299031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a4c24-19d9-41d4-8c5d-525aee09713c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.299494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a4c24-19d9-41d4-8c5d-525aee09713c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.299980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a4c24-19d9-41d4-8c5d-525aee09713c admin admin] PATH_INFO: `/v3/groups/2b357f7d938a4122998fc0f75fea508a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.310082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a4c24-19d9-41d4-8c5d-525aee09713c admin admin] RBAC: Authorizing `identity:get_group(group_id=2b357f7d938a4122998fc0f75fea508a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:21.312147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WARNING py.warnings [None req-d84a4c24-19d9-41d4-8c5d-525aee09713c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:42:21.312147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: warnings.warn(msg) Oct 14 17:42:21.312147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: Oct 14 17:42:21.313707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a4c24-19d9-41d4-8c5d-525aee09713c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:21.315518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 136/262] 10.222.0.27 () {56 vars in 1201 bytes} [Fri Oct 14 17:42:21 2022] GET /identity/v3/groups/2b357f7d938a4122998fc0f75fea508a => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:21.321977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf1d16-8788-4d01-b87a-9b08e27f0ce3 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:21.332379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf1d16-8788-4d01-b87a-9b08e27f0ce3 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:21.337932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e94a9-a33e-4c8c-bca5-d757b8e34ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:21.338781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e94a9-a33e-4c8c-bca5-d757b8e34ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.339246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e94a9-a33e-4c8c-bca5-d757b8e34ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.339719 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e94a9-a33e-4c8c-bca5-d757b8e34ec1 admin admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.349540 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e94a9-a33e-4c8c-bca5-d757b8e34ec1 admin admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:21.353010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e94a9-a33e-4c8c-bca5-d757b8e34ec1 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:21.354913 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 127/263] 10.222.0.27 () {56 vars in 1207 bytes} [Fri Oct 14 17:42:21 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:21.360918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a4c24-19d9-41d4-8c5d-525aee09713c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:21.373227 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a4c24-19d9-41d4-8c5d-525aee09713c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:21.384456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da694f86-a883-49e0-8e05-0cee9baf1615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:21.385172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694f86-a883-49e0-8e05-0cee9baf1615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.385451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694f86-a883-49e0-8e05-0cee9baf1615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.385726 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694f86-a883-49e0-8e05-0cee9baf1615 admin admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/groups/2b357f7d938a4122998fc0f75fea508a/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.389072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da694f86-a883-49e0-8e05-0cee9baf1615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0511b004c94f1bbed48dd4fd34cb6a, group_id=2b357f7d938a4122998fc0f75fea508a, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:21.390339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WARNING py.warnings [None req-da694f86-a883-49e0-8e05-0cee9baf1615 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:42:21.390339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: warnings.warn(msg) Oct 14 17:42:21.390339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: Oct 14 17:42:21.392726 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da694f86-a883-49e0-8e05-0cee9baf1615 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:21.422375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 137/264] 10.222.0.27 () {58 vars in 1463 bytes} [Fri Oct 14 17:42:21 2022] PUT /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/groups/2b357f7d938a4122998fc0f75fea508a/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:42:21.544549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e94a9-a33e-4c8c-bca5-d757b8e34ec1 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:21.556862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e94a9-a33e-4c8c-bca5-d757b8e34ec1 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:21.631380 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696365bb-a44d-49a8-8093-410813dd3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:21.632158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696365bb-a44d-49a8-8093-410813dd3be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.632533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696365bb-a44d-49a8-8093-410813dd3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.632812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696365bb-a44d-49a8-8093-410813dd3be5 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.649233 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696365bb-a44d-49a8-8093-410813dd3be5 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-696365bb-a44d-49a8-8093-410813dd3be5 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:21.654234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:21.655114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:21.655917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 128/265] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Oct 14 17:42:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 112 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:21.661520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da694f86-a883-49e0-8e05-0cee9baf1615 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:21.683542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da694f86-a883-49e0-8e05-0cee9baf1615 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:21.692734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503d591-a5bc-4f5b-8c54-2cefb655e175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:21.693362 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503d591-a5bc-4f5b-8c54-2cefb655e175 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.693636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503d591-a5bc-4f5b-8c54-2cefb655e175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.694709 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503d591-a5bc-4f5b-8c54-2cefb655e175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.696608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503d591-a5bc-4f5b-8c54-2cefb655e175 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:21.696908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503d591-a5bc-4f5b-8c54-2cefb655e175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:21.697930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WARNING py.warnings [None req-0503d591-a5bc-4f5b-8c54-2cefb655e175 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:42:21.697930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: warnings.warn(msg) Oct 14 17:42:21.697930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: Oct 14 17:42:21.698838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503d591-a5bc-4f5b-8c54-2cefb655e175 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:21.707102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8ee2d-36dd-4f75-a1a2-6b0d1756bfb6 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.707392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8ee2d-36dd-4f75-a1a2-6b0d1756bfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.707672 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8ee2d-36dd-4f75-a1a2-6b0d1756bfb6 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.709289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 129/266] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:21.711540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 138/267] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Oct 14 17:42:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:21.717301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503d591-a5bc-4f5b-8c54-2cefb655e175 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:21.718231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a734be-ee6f-4af6-b5c4-c8e008e1704e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.719059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a734be-ee6f-4af6-b5c4-c8e008e1704e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.720199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a734be-ee6f-4af6-b5c4-c8e008e1704e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.738100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503d591-a5bc-4f5b-8c54-2cefb655e175 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:21.751674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00d168-f5e4-4241-a83f-bb62a6ac706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:21.752621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00d168-f5e4-4241-a83f-bb62a6ac706b admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.753037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00d168-f5e4-4241-a83f-bb62a6ac706b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.753489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00d168-f5e4-4241-a83f-bb62a6ac706b admin admin] PATH_INFO: `/v3/groups/2b357f7d938a4122998fc0f75fea508a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.755079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00d168-f5e4-4241-a83f-bb62a6ac706b admin admin] RBAC: Authorizing `identity:get_group(group_id=2b357f7d938a4122998fc0f75fea508a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:21.759853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00d168-f5e4-4241-a83f-bb62a6ac706b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:21.765498 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 139/268] 10.222.0.27 () {56 vars in 1201 bytes} [Fri Oct 14 17:42:21 2022] GET /identity/v3/groups/2b357f7d938a4122998fc0f75fea508a => generated 226 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:21.768676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-63a734be-ee6f-4af6-b5c4-c8e008e1704e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:21.770216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-63a734be-ee6f-4af6-b5c4-c8e008e1704e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:21.770985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00d168-f5e4-4241-a83f-bb62a6ac706b admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:21.779274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 130/269] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 14 17:42:21.786242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00d168-f5e4-4241-a83f-bb62a6ac706b admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:21.791685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aefbdc-8a2d-4446-82d0-15af3856c74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:21.792576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aefbdc-8a2d-4446-82d0-15af3856c74a admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.793010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aefbdc-8a2d-4446-82d0-15af3856c74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.793451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aefbdc-8a2d-4446-82d0-15af3856c74a admin admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.794854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aefbdc-8a2d-4446-82d0-15af3856c74a admin admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:21.797250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aefbdc-8a2d-4446-82d0-15af3856c74a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:21.799440 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 140/270] 10.222.0.27 () {56 vars in 1207 bytes} [Fri Oct 14 17:42:21 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:21.805805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a734be-ee6f-4af6-b5c4-c8e008e1704e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:21.815675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a734be-ee6f-4af6-b5c4-c8e008e1704e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:21.821748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adba5a1-a23b-40cb-bdc6-fcd5d257fd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:21.822594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adba5a1-a23b-40cb-bdc6-fcd5d257fd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:21.823027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adba5a1-a23b-40cb-bdc6-fcd5d257fd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:21.823488 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adba5a1-a23b-40cb-bdc6-fcd5d257fd91 admin admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/groups/2b357f7d938a4122998fc0f75fea508a/roles/1cc2ea78f84c4491967839cfead3fd26` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:21.825998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adba5a1-a23b-40cb-bdc6-fcd5d257fd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0511b004c94f1bbed48dd4fd34cb6a, group_id=2b357f7d938a4122998fc0f75fea508a, role_id=1cc2ea78f84c4491967839cfead3fd26)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:21.827279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: WARNING py.warnings [None req-0adba5a1-a23b-40cb-bdc6-fcd5d257fd91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:42:21.827279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: warnings.warn(msg) Oct 14 17:42:21.827279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: Oct 14 17:42:21.829600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adba5a1-a23b-40cb-bdc6-fcd5d257fd91 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:21.845906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 131/271] 10.222.0.27 () {58 vars in 1463 bytes} [Fri Oct 14 17:42:21 2022] PUT /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a/groups/2b357f7d938a4122998fc0f75fea508a/roles/1cc2ea78f84c4491967839cfead3fd26 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:42:22.201919 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08497e98-9c4f-4e53-844a-1edc44c44226 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:22.202404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08497e98-9c4f-4e53-844a-1edc44c44226 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:22.202825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08497e98-9c4f-4e53-844a-1edc44c44226 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:22.204626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 141/272] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:22.214820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc77bd-8081-492d-84ec-9a5da46bafc8 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:22.215352 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc77bd-8081-492d-84ec-9a5da46bafc8 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:22.215814 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc77bd-8081-492d-84ec-9a5da46bafc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:22.244984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-70cc77bd-8081-492d-84ec-9a5da46bafc8 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:22.246531 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-70cc77bd-8081-492d-84ec-9a5da46bafc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:22.249955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:22.336108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 132/273] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:22.338499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08497e98-9c4f-4e53-844a-1edc44c44226 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:22.361505 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:22.435145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08497e98-9c4f-4e53-844a-1edc44c44226 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:22.439634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198d794-2a57-43a0-a556-f8b4a939f96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:22.440287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198d794-2a57-43a0-a556-f8b4a939f96c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:22.440559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198d794-2a57-43a0-a556-f8b4a939f96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:22.440823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198d794-2a57-43a0-a556-f8b4a939f96c admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:22.457164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5198d794-2a57-43a0-a556-f8b4a939f96c admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-5198d794-2a57-43a0-a556-f8b4a939f96c admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:22.463417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:22.464335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:22.465129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 142/274] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Oct 14 17:42:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 127 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:22.471713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc77bd-8081-492d-84ec-9a5da46bafc8 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:22.485261 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc77bd-8081-492d-84ec-9a5da46bafc8 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:22.496107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5052b8-b34a-45c7-ab2c-a0195f048c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:22.496726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5052b8-b34a-45c7-ab2c-a0195f048c14 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:22.497040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5052b8-b34a-45c7-ab2c-a0195f048c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:22.497309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5052b8-b34a-45c7-ab2c-a0195f048c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:22.497977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5052b8-b34a-45c7-ab2c-a0195f048c14 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:22.498271 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5052b8-b34a-45c7-ab2c-a0195f048c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:22.499863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5052b8-b34a-45c7-ab2c-a0195f048c14 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:22.510557 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 133/275] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:42:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:22.516476 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198d794-2a57-43a0-a556-f8b4a939f96c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:22.527510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198d794-2a57-43a0-a556-f8b4a939f96c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:22.532182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c55f78-24d5-4964-9e88-9250179f5a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:22.532884 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c55f78-24d5-4964-9e88-9250179f5a2b admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:22.533162 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c55f78-24d5-4964-9e88-9250179f5a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:22.533430 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c55f78-24d5-4964-9e88-9250179f5a2b admin admin] PATH_INFO: `/v3/groups/2b357f7d938a4122998fc0f75fea508a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:22.537269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c55f78-24d5-4964-9e88-9250179f5a2b admin admin] RBAC: Authorizing `identity:get_group(group_id=2b357f7d938a4122998fc0f75fea508a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:22.540274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c55f78-24d5-4964-9e88-9250179f5a2b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:22.542464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 143/276] 10.222.0.27 () {56 vars in 1201 bytes} [Fri Oct 14 17:42:22 2022] GET /identity/v3/groups/2b357f7d938a4122998fc0f75fea508a => generated 226 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:22.547640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5052b8-b34a-45c7-ab2c-a0195f048c14 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:22.570230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5052b8-b34a-45c7-ab2c-a0195f048c14 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:22.580302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69695a-1786-4b2e-a98f-f376452f5ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:22.580956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69695a-1786-4b2e-a98f-f376452f5ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:22.581231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69695a-1786-4b2e-a98f-f376452f5ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:22.581504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69695a-1786-4b2e-a98f-f376452f5ca2 admin admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:22.588656 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69695a-1786-4b2e-a98f-f376452f5ca2 admin admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:22.590740 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69695a-1786-4b2e-a98f-f376452f5ca2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:22.592741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 134/277] 10.222.0.27 () {56 vars in 1207 bytes} [Fri Oct 14 17:42:22 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:22.598386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c55f78-24d5-4964-9e88-9250179f5a2b admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:22.613152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c55f78-24d5-4964-9e88-9250179f5a2b admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:22.621859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99030e56-a3c8-4371-b9b6-d862032e3bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:22.622675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99030e56-a3c8-4371-b9b6-d862032e3bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:22.622953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99030e56-a3c8-4371-b9b6-d862032e3bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:22.623221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99030e56-a3c8-4371-b9b6-d862032e3bd6 admin admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/groups/2b357f7d938a4122998fc0f75fea508a/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:22.626891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99030e56-a3c8-4371-b9b6-d862032e3bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5d8b8f16a64419ab27d85700e9bdb5, group_id=2b357f7d938a4122998fc0f75fea508a, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:22.629709 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99030e56-a3c8-4371-b9b6-d862032e3bd6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:22.658091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 144/278] 10.222.0.27 () {58 vars in 1463 bytes} [Fri Oct 14 17:42:22 2022] PUT /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/groups/2b357f7d938a4122998fc0f75fea508a/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:22.789614 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69695a-1786-4b2e-a98f-f376452f5ca2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:22.821576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c1864-8baa-4465-ab91-fe67bb02f2c7 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:22.822225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c1864-8baa-4465-ab91-fe67bb02f2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:22.822491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c1864-8baa-4465-ab91-fe67bb02f2c7 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:22.828026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 145/279] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:22.840977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b19e1e-a286-45f3-9e16-0110ffc6436c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:22.841253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b19e1e-a286-45f3-9e16-0110ffc6436c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:22.841505 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b19e1e-a286-45f3-9e16-0110ffc6436c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:22.878187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-35b19e1e-a286-45f3-9e16-0110ffc6436c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:22.879547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-35b19e1e-a286-45f3-9e16-0110ffc6436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:22.985828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 146/280] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 14 17:42:23.006800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69695a-1786-4b2e-a98f-f376452f5ca2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.054721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:23.069862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e31d4-98bb-4444-b73b-ea0c68cfc7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.070516 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e31d4-98bb-4444-b73b-ea0c68cfc7ad None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.070793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e31d4-98bb-4444-b73b-ea0c68cfc7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.071063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e31d4-98bb-4444-b73b-ea0c68cfc7ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.089431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e31d4-98bb-4444-b73b-ea0c68cfc7ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-bb1e31d4-98bb-4444-b73b-ea0c68cfc7ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:23.094916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:23.095865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:23.096698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 135/281] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:23.102486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b19e1e-a286-45f3-9e16-0110ffc6436c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.103473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901f8a8-fdea-45cd-94b3-5fc8c050ac0a None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.103746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901f8a8-fdea-45cd-94b3-5fc8c050ac0a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.104055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901f8a8-fdea-45cd-94b3-5fc8c050ac0a None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.105643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 136/282] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:42:23.111284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe10a58-6429-4fc7-b937-6e65d6b2b089 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.111559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe10a58-6429-4fc7-b937-6e65d6b2b089 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.111832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe10a58-6429-4fc7-b937-6e65d6b2b089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.140313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-9fe10a58-6429-4fc7-b937-6e65d6b2b089 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:23.141565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9fe10a58-6429-4fc7-b937-6e65d6b2b089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:23.158011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 137/283] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 14 17:42:23.242440 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:23.271917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b19e1e-a286-45f3-9e16-0110ffc6436c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.316226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d14d9-425f-43ac-8a0d-744deffc8b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.316754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d14d9-425f-43ac-8a0d-744deffc8b01 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.317023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d14d9-425f-43ac-8a0d-744deffc8b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.317285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d14d9-425f-43ac-8a0d-744deffc8b01 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.317951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d14d9-425f-43ac-8a0d-744deffc8b01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:23.318243 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d14d9-425f-43ac-8a0d-744deffc8b01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:23.320878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d14d9-425f-43ac-8a0d-744deffc8b01 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:23.329509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 147/284] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:23.335911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe10a58-6429-4fc7-b937-6e65d6b2b089 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.429647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d14d9-425f-43ac-8a0d-744deffc8b01 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.445527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d14d9-425f-43ac-8a0d-744deffc8b01 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.454949 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee58ec-8fe8-4139-b7c7-b3c0fc318852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.455731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee58ec-8fe8-4139-b7c7-b3c0fc318852 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.456058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee58ec-8fe8-4139-b7c7-b3c0fc318852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.456323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee58ec-8fe8-4139-b7c7-b3c0fc318852 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.472075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee58ec-8fe8-4139-b7c7-b3c0fc318852 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-e2ee58ec-8fe8-4139-b7c7-b3c0fc318852 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:23.478977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:23.479902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:23.480710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 148/285] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:23.487591 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee58ec-8fe8-4139-b7c7-b3c0fc318852 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.501520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee58ec-8fe8-4139-b7c7-b3c0fc318852 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.511331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67be167-fc1c-4e29-be08-2de1444a0b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.512387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67be167-fc1c-4e29-be08-2de1444a0b47 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.512658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67be167-fc1c-4e29-be08-2de1444a0b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.512965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67be167-fc1c-4e29-be08-2de1444a0b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.513845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67be167-fc1c-4e29-be08-2de1444a0b47 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:23.514196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67be167-fc1c-4e29-be08-2de1444a0b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:23.515902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67be167-fc1c-4e29-be08-2de1444a0b47 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:23.533220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 149/286] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:23.535293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe10a58-6429-4fc7-b937-6e65d6b2b089 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.542796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67be167-fc1c-4e29-be08-2de1444a0b47 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.553396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67be167-fc1c-4e29-be08-2de1444a0b47 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.564515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a50969-3540-4830-a474-2d9e58f2958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.565299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a50969-3540-4830-a474-2d9e58f2958f admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.565636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a50969-3540-4830-a474-2d9e58f2958f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.565906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a50969-3540-4830-a474-2d9e58f2958f admin admin] PATH_INFO: `/v3/groups/2b357f7d938a4122998fc0f75fea508a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.568622 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a50969-3540-4830-a474-2d9e58f2958f admin admin] RBAC: Authorizing `identity:get_group(group_id=2b357f7d938a4122998fc0f75fea508a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:23.571012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a50969-3540-4830-a474-2d9e58f2958f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:23.576985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 150/287] 10.222.0.27 () {56 vars in 1201 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/groups/2b357f7d938a4122998fc0f75fea508a => generated 226 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:23.582971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a50969-3540-4830-a474-2d9e58f2958f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.596962 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a50969-3540-4830-a474-2d9e58f2958f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.609586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391030b-e495-4aec-bfa2-cf7461fe8aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.610222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391030b-e495-4aec-bfa2-cf7461fe8aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.610493 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391030b-e495-4aec-bfa2-cf7461fe8aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.610759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391030b-e495-4aec-bfa2-cf7461fe8aa9 admin admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.616770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391030b-e495-4aec-bfa2-cf7461fe8aa9 admin admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:23.618921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391030b-e495-4aec-bfa2-cf7461fe8aa9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:23.620756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 151/288] 10.222.0.27 () {56 vars in 1207 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:23.625030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7f52-9e31-469d-ba9e-a5b9633e2504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.625841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7f52-9e31-469d-ba9e-a5b9633e2504 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.626098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7f52-9e31-469d-ba9e-a5b9633e2504 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.626373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7f52-9e31-469d-ba9e-a5b9633e2504 None admin] PATH_INFO: `/v3/users/0e2a1a82c5c84eb7a4b073838d953bab` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.626585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391030b-e495-4aec-bfa2-cf7461fe8aa9 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.628097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7f52-9e31-469d-ba9e-a5b9633e2504 None admin] RBAC: Authorizing `identity:get_user(user_id=0e2a1a82c5c84eb7a4b073838d953bab)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:23.632301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7f52-9e31-469d-ba9e-a5b9633e2504 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:23.637577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 138/289] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/users/0e2a1a82c5c84eb7a4b073838d953bab => generated 293 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:23.643368 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7f52-9e31-469d-ba9e-a5b9633e2504 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.652434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391030b-e495-4aec-bfa2-cf7461fe8aa9 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.662087 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51583476-68de-4e59-a327-16a4f2590347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.662888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583476-68de-4e59-a327-16a4f2590347 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.663202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583476-68de-4e59-a327-16a4f2590347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.663441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583476-68de-4e59-a327-16a4f2590347 admin admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/groups/2b357f7d938a4122998fc0f75fea508a/roles/1cc2ea78f84c4491967839cfead3fd26` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.667934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51583476-68de-4e59-a327-16a4f2590347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5d8b8f16a64419ab27d85700e9bdb5, group_id=2b357f7d938a4122998fc0f75fea508a, role_id=1cc2ea78f84c4491967839cfead3fd26)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:23.672853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51583476-68de-4e59-a327-16a4f2590347 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:23.705972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 152/290] 10.222.0.27 () {58 vars in 1463 bytes} [Fri Oct 14 17:42:23 2022] PUT /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5/groups/2b357f7d938a4122998fc0f75fea508a/roles/1cc2ea78f84c4491967839cfead3fd26 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:42:23.706896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51583476-68de-4e59-a327-16a4f2590347 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.752283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:23.825164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51583476-68de-4e59-a327-16a4f2590347 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.841860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d7fb3-d4fb-46de-9cd1-978c2037dd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.842503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7fb3-d4fb-46de-9cd1-978c2037dd7e admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.842771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7fb3-d4fb-46de-9cd1-978c2037dd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.843034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7fb3-d4fb-46de-9cd1-978c2037dd7e admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.872442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d7fb3-d4fb-46de-9cd1-978c2037dd7e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-4f7d7fb3-d4fb-46de-9cd1-978c2037dd7e admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:23.877385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:23.879690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:23.881985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 153/291] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:23.883551 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7f52-9e31-469d-ba9e-a5b9633e2504 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.890307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d7fb3-d4fb-46de-9cd1-978c2037dd7e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.910150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d7fb3-d4fb-46de-9cd1-978c2037dd7e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.923783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba68fc6-8cd1-431e-b822-8d8c3146f64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.924491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba68fc6-8cd1-431e-b822-8d8c3146f64c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.924766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba68fc6-8cd1-431e-b822-8d8c3146f64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.925083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba68fc6-8cd1-431e-b822-8d8c3146f64c admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.925838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba68fc6-8cd1-431e-b822-8d8c3146f64c admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:23.926145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba68fc6-8cd1-431e-b822-8d8c3146f64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:23.927841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba68fc6-8cd1-431e-b822-8d8c3146f64c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:23.946746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 154/292] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:23.952863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba68fc6-8cd1-431e-b822-8d8c3146f64c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.969072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba68fc6-8cd1-431e-b822-8d8c3146f64c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:23.976452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efbfcf-1e75-44cb-ab99-6366a183cb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.977159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efbfcf-1e75-44cb-ab99-6366a183cb1a admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.977436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efbfcf-1e75-44cb-ab99-6366a183cb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.977705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efbfcf-1e75-44cb-ab99-6366a183cb1a admin admin] PATH_INFO: `/v3/groups/bc1ea5c92b864a26bc18658925ce010b` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.985766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efbfcf-1e75-44cb-ab99-6366a183cb1a admin admin] RBAC: Authorizing `identity:get_group(group_id=bc1ea5c92b864a26bc18658925ce010b)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:23.987775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efbfcf-1e75-44cb-ab99-6366a183cb1a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:23.991141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 155/293] 10.222.0.27 () {56 vars in 1201 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/groups/bc1ea5c92b864a26bc18658925ce010b => generated 229 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:23.993700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54330818-d40d-4ec0-8e3e-a7f5187156af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:23.994325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54330818-d40d-4ec0-8e3e-a7f5187156af None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:23.994593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54330818-d40d-4ec0-8e3e-a7f5187156af None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:23.994854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54330818-d40d-4ec0-8e3e-a7f5187156af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:23.995625 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54330818-d40d-4ec0-8e3e-a7f5187156af None admin] RBAC: Adding query filter params (role.id=6d101684527d45fabea90eeb60fc3e99, user.id=0e2a1a82c5c84eb7a4b073838d953bab, scope.system=all) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:23.996168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54330818-d40d-4ec0-8e3e-a7f5187156af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:23.996871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efbfcf-1e75-44cb-ab99-6366a183cb1a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:23.997609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54330818-d40d-4ec0-8e3e-a7f5187156af None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:24.008908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efbfcf-1e75-44cb-ab99-6366a183cb1a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:24.014297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb24ad-0e99-45ac-86f8-5cf0a8b50b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:24.015247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb24ad-0e99-45ac-86f8-5cf0a8b50b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:24.015618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb24ad-0e99-45ac-86f8-5cf0a8b50b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:24.015958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb24ad-0e99-45ac-86f8-5cf0a8b50b55 admin admin] PATH_INFO: `/v3/projects/a41cbe9ae3574988a4c391824f7f340a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:24.017681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb24ad-0e99-45ac-86f8-5cf0a8b50b55 admin admin] RBAC: Authorizing `identity:get_project(project_id=a41cbe9ae3574988a4c391824f7f340a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:24.019630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb24ad-0e99-45ac-86f8-5cf0a8b50b55 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:24.024534 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 156/294] 10.222.0.27 () {56 vars in 1207 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/projects/a41cbe9ae3574988a4c391824f7f340a => generated 344 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:24.026280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 139/295] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Oct 14 17:42:23 2022] GET /identity/v3/role_assignments?user.id=0e2a1a82c5c84eb7a4b073838d953bab&scope.system=all&role.id=6d101684527d45fabea90eeb60fc3e99 => generated 510 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:24.033074 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb24ad-0e99-45ac-86f8-5cf0a8b50b55 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:24.047944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb24ad-0e99-45ac-86f8-5cf0a8b50b55 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:24.054795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60071ba0-3a05-4b9e-8c29-4708345e7fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:24.055591 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60071ba0-3a05-4b9e-8c29-4708345e7fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:24.055900 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60071ba0-3a05-4b9e-8c29-4708345e7fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:24.056306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60071ba0-3a05-4b9e-8c29-4708345e7fa9 admin admin] PATH_INFO: `/v3/projects/a41cbe9ae3574988a4c391824f7f340a/groups/bc1ea5c92b864a26bc18658925ce010b/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:24.058989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60071ba0-3a05-4b9e-8c29-4708345e7fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41cbe9ae3574988a4c391824f7f340a, group_id=bc1ea5c92b864a26bc18658925ce010b, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:24.062132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60071ba0-3a05-4b9e-8c29-4708345e7fa9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:24.080337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 157/296] 10.222.0.27 () {58 vars in 1463 bytes} [Fri Oct 14 17:42:24 2022] PUT /identity/v3/projects/a41cbe9ae3574988a4c391824f7f340a/groups/bc1ea5c92b864a26bc18658925ce010b/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:25.035341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a470e1-038a-4d3b-ad20-6a09c76da616 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:25.036178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a470e1-038a-4d3b-ad20-6a09c76da616 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:25.036178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a470e1-038a-4d3b-ad20-6a09c76da616 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:25.037486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 140/297] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:25.043602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a364744-1279-4443-9bd1-51e77dbc7747 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:25.044283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a364744-1279-4443-9bd1-51e77dbc7747 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:25.044283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a364744-1279-4443-9bd1-51e77dbc7747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:25.069324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9a364744-1279-4443-9bd1-51e77dbc7747 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:25.070810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9a364744-1279-4443-9bd1-51e77dbc7747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:25.073324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:25.160583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 158/298] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:25.168518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496917b-61a6-4fb6-972f-5786dba75c88 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:25.169066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496917b-61a6-4fb6-972f-5786dba75c88 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:25.169525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496917b-61a6-4fb6-972f-5786dba75c88 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:25.171023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 141/299] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:25.176352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec77c8-80b9-448e-9b90-6a39d013f7fb None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:25.176636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec77c8-80b9-448e-9b90-6a39d013f7fb None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:25.176905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec77c8-80b9-448e-9b90-6a39d013f7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:25.197587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-cdec77c8-80b9-448e-9b90-6a39d013f7fb None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:25.198758 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cdec77c8-80b9-448e-9b90-6a39d013f7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:25.276421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 159/300] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:25.570456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496917b-61a6-4fb6-972f-5786dba75c88 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:25.663792 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec77c8-80b9-448e-9b90-6a39d013f7fb None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:25.691630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496917b-61a6-4fb6-972f-5786dba75c88 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:25.720245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:25.737236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3b111-f845-4138-b281-f6eacd4996d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:25.737883 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3b111-f845-4138-b281-f6eacd4996d9 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:25.738158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3b111-f845-4138-b281-f6eacd4996d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:25.738425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3b111-f845-4138-b281-f6eacd4996d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:25.750345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3b111-f845-4138-b281-f6eacd4996d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-59f3b111-f845-4138-b281-f6eacd4996d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:25.755179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:25.756080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:25.756925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:25.757968 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 142/301] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:42:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:25.762479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3b111-f845-4138-b281-f6eacd4996d9 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:25.784727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:25.877088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec77c8-80b9-448e-9b90-6a39d013f7fb None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:25.887667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3b111-f845-4138-b281-f6eacd4996d9 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:25.926134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd477c43-0ed0-459b-9c4e-05b453e374cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:25.926663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd477c43-0ed0-459b-9c4e-05b453e374cc None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:25.926936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd477c43-0ed0-459b-9c4e-05b453e374cc None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:25.927201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd477c43-0ed0-459b-9c4e-05b453e374cc None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:25.927854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd477c43-0ed0-459b-9c4e-05b453e374cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:25.928198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd477c43-0ed0-459b-9c4e-05b453e374cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:25.930980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd477c43-0ed0-459b-9c4e-05b453e374cc None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:25.939447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 143/302] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:25.945499 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd477c43-0ed0-459b-9c4e-05b453e374cc None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:25.976630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b673d4-1458-4708-b390-da84db3eedbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:25.977287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b673d4-1458-4708-b390-da84db3eedbb None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:25.977567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b673d4-1458-4708-b390-da84db3eedbb None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:25.977834 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b673d4-1458-4708-b390-da84db3eedbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:25.978302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:26.008576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b673d4-1458-4708-b390-da84db3eedbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-78b673d4-1458-4708-b390-da84db3eedbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:26.012951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:26.013822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:26.014633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 160/303] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:26.025567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b673d4-1458-4708-b390-da84db3eedbb None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:26.082793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd477c43-0ed0-459b-9c4e-05b453e374cc None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:26.147146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1a264-9706-4b7b-b1a2-4136bcf2f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:26.148142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1a264-9706-4b7b-b1a2-4136bcf2f68d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:26.148633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1a264-9706-4b7b-b1a2-4136bcf2f68d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:26.149090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1a264-9706-4b7b-b1a2-4136bcf2f68d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:26.186157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b673d4-1458-4708-b390-da84db3eedbb None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:26.222390 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:26.231789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2cb5b-477f-40ba-901b-03da7a7a1dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:26.232996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2cb5b-477f-40ba-901b-03da7a7a1dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:26.233564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2cb5b-477f-40ba-901b-03da7a7a1dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:26.234043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2cb5b-477f-40ba-901b-03da7a7a1dc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:26.235112 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2cb5b-477f-40ba-901b-03da7a7a1dc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:26.235551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2cb5b-477f-40ba-901b-03da7a7a1dc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:26.237664 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2cb5b-477f-40ba-901b-03da7a7a1dc3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:26.250690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 161/304] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:26.255655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2cb5b-477f-40ba-901b-03da7a7a1dc3 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:26.281594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1a264-9706-4b7b-b1a2-4136bcf2f68d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:26.283177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1a264-9706-4b7b-b1a2-4136bcf2f68d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:26.317005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:26.391767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2cb5b-477f-40ba-901b-03da7a7a1dc3 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:26.424483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 144/305] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 14 17:42:26.425837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894255c-0954-47d4-bc0e-7ededcaa235a None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:26.426125 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894255c-0954-47d4-bc0e-7ededcaa235a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:26.426398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894255c-0954-47d4-bc0e-7ededcaa235a None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:26.427645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 145/306] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:26.433723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894255c-0954-47d4-bc0e-7ededcaa235a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:26.442716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c3bac-f4a3-4ca3-b744-3fab0d45c024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:26.443511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c3bac-f4a3-4ca3-b744-3fab0d45c024 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:26.445529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c3bac-f4a3-4ca3-b744-3fab0d45c024 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:26.445955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c3bac-f4a3-4ca3-b744-3fab0d45c024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:26.517481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:26.535581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c3bac-f4a3-4ca3-b744-3fab0d45c024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:26.537722 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c3bac-f4a3-4ca3-b744-3fab0d45c024 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:26.563847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894255c-0954-47d4-bc0e-7ededcaa235a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:26.599302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:26.611539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe390bf3-5c34-421f-9d9a-c6e357306362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:26.612544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe390bf3-5c34-421f-9d9a-c6e357306362 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:26.613238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe390bf3-5c34-421f-9d9a-c6e357306362 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:26.613643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe390bf3-5c34-421f-9d9a-c6e357306362 None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:26.614940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe390bf3-5c34-421f-9d9a-c6e357306362 None admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:26.616618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe390bf3-5c34-421f-9d9a-c6e357306362 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:26.619555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 146/307] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:26 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:26.621070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae383e4-210f-4f51-8267-434cdd65eb87 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:26.621489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae383e4-210f-4f51-8267-434cdd65eb87 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:26.621899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae383e4-210f-4f51-8267-434cdd65eb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:26.643337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-eae383e4-210f-4f51-8267-434cdd65eb87 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:26.647938 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-eae383e4-210f-4f51-8267-434cdd65eb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:26.673945 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 162/308] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:26.674888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c3bac-f4a3-4ca3-b744-3fab0d45c024 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:26.738273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 147/309] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:26.739504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae383e4-210f-4f51-8267-434cdd65eb87 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:26.798094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:26.809209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c3bac-f4a3-4ca3-b744-3fab0d45c024 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:26.853972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb15fc8-b270-4ee4-acf0-d456ea828281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:26.854878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb15fc8-b270-4ee4-acf0-d456ea828281 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:26.855159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb15fc8-b270-4ee4-acf0-d456ea828281 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:26.855432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb15fc8-b270-4ee4-acf0-d456ea828281 None admin] PATH_INFO: `/v3/groups/2b357f7d938a4122998fc0f75fea508a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:26.856806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb15fc8-b270-4ee4-acf0-d456ea828281 None admin] RBAC: Authorizing `identity:get_group(group_id=2b357f7d938a4122998fc0f75fea508a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:26.858936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb15fc8-b270-4ee4-acf0-d456ea828281 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:26.861854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 163/310] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Oct 14 17:42:26 2022] GET /identity/v3/groups/2b357f7d938a4122998fc0f75fea508a => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:26.862304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fa17e-065d-4c2d-aadc-8af6749e9857 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:26.862580 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fa17e-065d-4c2d-aadc-8af6749e9857 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:26.862847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fa17e-065d-4c2d-aadc-8af6749e9857 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:26.864826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 164/311] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:26.872569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae383e4-210f-4f51-8267-434cdd65eb87 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:26.874091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8dd7a-5ce4-43dc-ac74-74786385ea6c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:26.876252 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8dd7a-5ce4-43dc-ac74-74786385ea6c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:26.876765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8dd7a-5ce4-43dc-ac74-74786385ea6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:26.906094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-80d8dd7a-5ce4-43dc-ac74-74786385ea6c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:26.907655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-80d8dd7a-5ce4-43dc-ac74-74786385ea6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:26.908675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:26.921649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b089fb4-f18f-48f6-84a5-9595b2d9a405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:26.922197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b089fb4-f18f-48f6-84a5-9595b2d9a405 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:26.922479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b089fb4-f18f-48f6-84a5-9595b2d9a405 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:26.922749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b089fb4-f18f-48f6-84a5-9595b2d9a405 None admin] PATH_INFO: `/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:26.924056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b089fb4-f18f-48f6-84a5-9595b2d9a405 None admin] RBAC: Authorizing `identity:get_project(project_id=6e0511b004c94f1bbed48dd4fd34cb6a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:26.925552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b089fb4-f18f-48f6-84a5-9595b2d9a405 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:26.927304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 148/312] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:26 2022] GET /identity/v3/projects/6e0511b004c94f1bbed48dd4fd34cb6a => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:26.928896 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b089fb4-f18f-48f6-84a5-9595b2d9a405 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:27.001069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 165/313] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:27.002143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8dd7a-5ce4-43dc-ac74-74786385ea6c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:27.071030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b089fb4-f18f-48f6-84a5-9595b2d9a405 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:27.109726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75df786-41f7-4ff9-b0c4-1a7e1e564d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:27.111183 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df786-41f7-4ff9-b0c4-1a7e1e564d8b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:27.111679 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df786-41f7-4ff9-b0c4-1a7e1e564d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:27.112234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df786-41f7-4ff9-b0c4-1a7e1e564d8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:27.112739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:27.113811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75df786-41f7-4ff9-b0c4-1a7e1e564d8b None admin] RBAC: Adding query filter params (group.id=2b357f7d938a4122998fc0f75fea508a, role.id=1cc2ea78f84c4491967839cfead3fd26, scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:27.114341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75df786-41f7-4ff9-b0c4-1a7e1e564d8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:27.116608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75df786-41f7-4ff9-b0c4-1a7e1e564d8b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:27.123361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 149/314] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Oct 14 17:42:26 2022] GET /identity/v3/role_assignments?group.id=2b357f7d938a4122998fc0f75fea508a&scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a&role.id=1cc2ea78f84c4491967839cfead3fd26 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:27.127610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8dd7a-5ce4-43dc-ac74-74786385ea6c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:27.167665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def09d26-695f-4fe7-afc7-f6a73479f312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:27.168354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def09d26-695f-4fe7-afc7-f6a73479f312 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:27.168624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def09d26-695f-4fe7-afc7-f6a73479f312 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:27.168888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def09d26-695f-4fe7-afc7-f6a73479f312 None admin] PATH_INFO: `/v3/groups/2b357f7d938a4122998fc0f75fea508a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:27.169984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def09d26-695f-4fe7-afc7-f6a73479f312 None admin] RBAC: Authorizing `identity:get_group(group_id=2b357f7d938a4122998fc0f75fea508a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:27.171429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def09d26-695f-4fe7-afc7-f6a73479f312 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:27.173019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 166/315] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Oct 14 17:42:27 2022] GET /identity/v3/groups/2b357f7d938a4122998fc0f75fea508a => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:27.179765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75df786-41f7-4ff9-b0c4-1a7e1e564d8b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:27.192943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def09d26-695f-4fe7-afc7-f6a73479f312 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:27.272293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:27.324397 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def09d26-695f-4fe7-afc7-f6a73479f312 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:27.344108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75df786-41f7-4ff9-b0c4-1a7e1e564d8b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:27.374800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96015d84-a371-4286-8926-4086ed484488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:27.375438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015d84-a371-4286-8926-4086ed484488 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:27.375710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015d84-a371-4286-8926-4086ed484488 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:27.376073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015d84-a371-4286-8926-4086ed484488 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:27.389683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96015d84-a371-4286-8926-4086ed484488 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-96015d84-a371-4286-8926-4086ed484488 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:27.394201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:27.395142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:27.395981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 167/316] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:27.397618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea67a1d-f1f8-4250-949a-079646a1ceb4 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:27.397618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea67a1d-f1f8-4250-949a-079646a1ceb4 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:27.397618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea67a1d-f1f8-4250-949a-079646a1ceb4 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:27.398700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 168/317] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:27.402040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c1ad5-8f70-4d3a-8414-b4b1ed03444a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:27.402175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea67a1d-f1f8-4250-949a-079646a1ceb4 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:27.403643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c1ad5-8f70-4d3a-8414-b4b1ed03444a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:27.404187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c1ad5-8f70-4d3a-8414-b4b1ed03444a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:27.404608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c1ad5-8f70-4d3a-8414-b4b1ed03444a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:27.405630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c1ad5-8f70-4d3a-8414-b4b1ed03444a None admin] RBAC: Adding query filter params (group.id=2b357f7d938a4122998fc0f75fea508a, role.id=9ab8d70da5894a2f99c6d22e5351baa3, scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:27.406081 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c1ad5-8f70-4d3a-8414-b4b1ed03444a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:27.407683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c1ad5-8f70-4d3a-8414-b4b1ed03444a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:27.416387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 150/318] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Oct 14 17:42:27 2022] GET /identity/v3/role_assignments?group.id=2b357f7d938a4122998fc0f75fea508a&scope.project.id=6e0511b004c94f1bbed48dd4fd34cb6a&role.id=9ab8d70da5894a2f99c6d22e5351baa3 => generated 611 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:27.417285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c1ad5-8f70-4d3a-8414-b4b1ed03444a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:27.438937 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:27.557295 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c1ad5-8f70-4d3a-8414-b4b1ed03444a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:27.563760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea67a1d-f1f8-4250-949a-079646a1ceb4 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:27.595370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:27.609032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86bd4b-8911-4c41-ac63-b08778c65a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:27.609653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86bd4b-8911-4c41-ac63-b08778c65a0f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:27.610451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86bd4b-8911-4c41-ac63-b08778c65a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:27.610451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86bd4b-8911-4c41-ac63-b08778c65a0f None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:27.610853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86bd4b-8911-4c41-ac63-b08778c65a0f None admin] RBAC: Adding query filter params (name=member) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:27.611168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86bd4b-8911-4c41-ac63-b08778c65a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:27.612559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86bd4b-8911-4c41-ac63-b08778c65a0f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:27.614673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33446de4-02c7-4625-8ce7-83bab592691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:27.616234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33446de4-02c7-4625-8ce7-83bab592691b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:27.616893 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33446de4-02c7-4625-8ce7-83bab592691b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:27.617354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33446de4-02c7-4625-8ce7-83bab592691b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:27.621879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 169/319] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:42:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:27.623158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac865-744c-467f-be83-f89d5cc223ea None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:27.623378 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac865-744c-467f-be83-f89d5cc223ea None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:27.623648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac865-744c-467f-be83-f89d5cc223ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:27.630407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33446de4-02c7-4625-8ce7-83bab592691b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-33446de4-02c7-4625-8ce7-83bab592691b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:27.637594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:27.638546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 151/320] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:42:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:27.641635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33446de4-02c7-4625-8ce7-83bab592691b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:27.648881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8b4ac865-744c-467f-be83-f89d5cc223ea None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:27.649679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8b4ac865-744c-467f-be83-f89d5cc223ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:27.737307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 170/321] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:27.738177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ac865-744c-467f-be83-f89d5cc223ea None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:27.748714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:27.766257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33446de4-02c7-4625-8ce7-83bab592691b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:27.805790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deeb8e5-e83e-4747-bf9d-f2da064319ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:27.806399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deeb8e5-e83e-4747-bf9d-f2da064319ea None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:27.806676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deeb8e5-e83e-4747-bf9d-f2da064319ea None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:27.806947 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deeb8e5-e83e-4747-bf9d-f2da064319ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:27.857949 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ac865-744c-467f-be83-f89d5cc223ea None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:27.868075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:27.890202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deeb8e5-e83e-4747-bf9d-f2da064319ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:27.891807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deeb8e5-e83e-4747-bf9d-f2da064319ea None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:27.906109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d993d7d-fe57-4f09-8441-7f068ceb6bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:27.906763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d993d7d-fe57-4f09-8441-7f068ceb6bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:27.907041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d993d7d-fe57-4f09-8441-7f068ceb6bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:27.907309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d993d7d-fe57-4f09-8441-7f068ceb6bf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:27.908934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d993d7d-fe57-4f09-8441-7f068ceb6bf4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:27.908934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d993d7d-fe57-4f09-8441-7f068ceb6bf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:27.911688 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d993d7d-fe57-4f09-8441-7f068ceb6bf4 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:27.922503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 171/322] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:27.928413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d993d7d-fe57-4f09-8441-7f068ceb6bf4 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.018163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:28.026765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 152/323] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:28.027620 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deeb8e5-e83e-4747-bf9d-f2da064319ea None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.059172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d993d7d-fe57-4f09-8441-7f068ceb6bf4 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:28.096560 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaab1fe-5644-4082-abbb-746111aa84ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:28.097178 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaab1fe-5644-4082-abbb-746111aa84ed None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:28.097454 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaab1fe-5644-4082-abbb-746111aa84ed None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:28.097718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaab1fe-5644-4082-abbb-746111aa84ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:28.127980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:28.150425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deeb8e5-e83e-4747-bf9d-f2da064319ea None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:28.174617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaab1fe-5644-4082-abbb-746111aa84ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:28.176124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaab1fe-5644-4082-abbb-746111aa84ed None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:28.193017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82904ce0-8b49-4b51-8e93-0260044255e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:28.193648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82904ce0-8b49-4b51-8e93-0260044255e3 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:28.194468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82904ce0-8b49-4b51-8e93-0260044255e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:28.194741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82904ce0-8b49-4b51-8e93-0260044255e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:28.206945 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82904ce0-8b49-4b51-8e93-0260044255e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-82904ce0-8b49-4b51-8e93-0260044255e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:28.212416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:28.213286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:28.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 153/324] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:42:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:28.214784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82904ce0-8b49-4b51-8e93-0260044255e3 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.257731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:28.295423 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 172/325] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:28.296297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaab1fe-5644-4082-abbb-746111aa84ed None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.327313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82904ce0-8b49-4b51-8e93-0260044255e3 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:28.371677 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5bf95-8e50-4c57-b8b6-4119a3f3e928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:28.373096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5bf95-8e50-4c57-b8b6-4119a3f3e928 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:28.373096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5bf95-8e50-4c57-b8b6-4119a3f3e928 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:28.373096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5bf95-8e50-4c57-b8b6-4119a3f3e928 None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:28.374088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5bf95-8e50-4c57-b8b6-4119a3f3e928 None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:28.375541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5bf95-8e50-4c57-b8b6-4119a3f3e928 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:28.377282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 154/326] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:28 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:28.378109 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5bf95-8e50-4c57-b8b6-4119a3f3e928 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.393726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:28.426867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaab1fe-5644-4082-abbb-746111aa84ed None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:28.470960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca086bc-a389-4b68-a799-204b86a37634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:28.471568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca086bc-a389-4b68-a799-204b86a37634 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:28.472349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca086bc-a389-4b68-a799-204b86a37634 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:28.472627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca086bc-a389-4b68-a799-204b86a37634 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:28.473294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca086bc-a389-4b68-a799-204b86a37634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:28.474551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca086bc-a389-4b68-a799-204b86a37634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:28.475914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca086bc-a389-4b68-a799-204b86a37634 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:28.485015 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 173/327] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:28.485954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca086bc-a389-4b68-a799-204b86a37634 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.505804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5bf95-8e50-4c57-b8b6-4119a3f3e928 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:28.520168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:28.550416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff456f-d167-464a-a464-422f4c53771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:28.551572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff456f-d167-464a-a464-422f4c53771f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:28.551572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff456f-d167-464a-a464-422f4c53771f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:28.551572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff456f-d167-464a-a464-422f4c53771f None admin] PATH_INFO: `/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:28.554315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff456f-d167-464a-a464-422f4c53771f None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d8b8f16a64419ab27d85700e9bdb5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:28.555510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff456f-d167-464a-a464-422f4c53771f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:28.557238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 155/328] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:28 2022] GET /identity/v3/projects/8a5d8b8f16a64419ab27d85700e9bdb5 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:28.557996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff456f-d167-464a-a464-422f4c53771f None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.598597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca086bc-a389-4b68-a799-204b86a37634 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:28.632428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03548d-d1b6-4512-8a50-1c5cc2b709b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:28.632982 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03548d-d1b6-4512-8a50-1c5cc2b709b3 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:28.633262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03548d-d1b6-4512-8a50-1c5cc2b709b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:28.633531 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03548d-d1b6-4512-8a50-1c5cc2b709b3 None admin] PATH_INFO: `/v3/groups/2b357f7d938a4122998fc0f75fea508a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:28.634648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03548d-d1b6-4512-8a50-1c5cc2b709b3 None admin] RBAC: Authorizing `identity:get_group(group_id=2b357f7d938a4122998fc0f75fea508a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:28.636114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03548d-d1b6-4512-8a50-1c5cc2b709b3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:28.637939 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 174/329] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Oct 14 17:42:28 2022] GET /identity/v3/groups/2b357f7d938a4122998fc0f75fea508a => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:28.638775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03548d-d1b6-4512-8a50-1c5cc2b709b3 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.639528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:28.661575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff456f-d167-464a-a464-422f4c53771f None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:28.695092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99860b89-dfe8-4fce-8718-2d65303c175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:28.695694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99860b89-dfe8-4fce-8718-2d65303c175e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:28.696005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99860b89-dfe8-4fce-8718-2d65303c175e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:28.696276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99860b89-dfe8-4fce-8718-2d65303c175e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:28.737322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:28.740285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03548d-d1b6-4512-8a50-1c5cc2b709b3 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:28.764050 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99860b89-dfe8-4fce-8718-2d65303c175e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:28.765502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99860b89-dfe8-4fce-8718-2d65303c175e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:28.775524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032ac8b-00c7-4056-bbee-d3641d65c74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:28.776180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032ac8b-00c7-4056-bbee-d3641d65c74e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:28.776454 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032ac8b-00c7-4056-bbee-d3641d65c74e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:28.776724 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032ac8b-00c7-4056-bbee-d3641d65c74e None admin] PATH_INFO: `/v3/groups/2b357f7d938a4122998fc0f75fea508a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:28.777818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032ac8b-00c7-4056-bbee-d3641d65c74e None admin] RBAC: Authorizing `identity:get_group(group_id=2b357f7d938a4122998fc0f75fea508a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:28.779262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032ac8b-00c7-4056-bbee-d3641d65c74e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:28.780831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 175/330] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Oct 14 17:42:28 2022] GET /identity/v3/groups/2b357f7d938a4122998fc0f75fea508a => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:28.781586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032ac8b-00c7-4056-bbee-d3641d65c74e None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.841576 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:28.863647 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 156/331] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:28.864488 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99860b89-dfe8-4fce-8718-2d65303c175e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.885753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032ac8b-00c7-4056-bbee-d3641d65c74e None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:28.918569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7b725-0d73-4e02-bbbd-373cc65ebf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:28.919157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7b725-0d73-4e02-bbbd-373cc65ebf34 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:28.919433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7b725-0d73-4e02-bbbd-373cc65ebf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:28.919695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7b725-0d73-4e02-bbbd-373cc65ebf34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:28.920437 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7b725-0d73-4e02-bbbd-373cc65ebf34 None admin] RBAC: Adding query filter params (group.id=2b357f7d938a4122998fc0f75fea508a, role.id=9ab8d70da5894a2f99c6d22e5351baa3, scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:28.920736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7b725-0d73-4e02-bbbd-373cc65ebf34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:28.922156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7b725-0d73-4e02-bbbd-373cc65ebf34 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:28.927669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 176/332] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Oct 14 17:42:28 2022] GET /identity/v3/role_assignments?group.id=2b357f7d938a4122998fc0f75fea508a&scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5&role.id=9ab8d70da5894a2f99c6d22e5351baa3 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:28.928572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7b725-0d73-4e02-bbbd-373cc65ebf34 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:28.964549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:28.985062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99860b89-dfe8-4fce-8718-2d65303c175e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:29.024547 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dfc8a-e439-4528-aa2e-caf69f2f670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:29.025161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dfc8a-e439-4528-aa2e-caf69f2f670b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:29.025431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dfc8a-e439-4528-aa2e-caf69f2f670b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:29.025698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dfc8a-e439-4528-aa2e-caf69f2f670b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:29.026422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dfc8a-e439-4528-aa2e-caf69f2f670b None admin] RBAC: Adding query filter params (group.id=2b357f7d938a4122998fc0f75fea508a, role.id=1cc2ea78f84c4491967839cfead3fd26, scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:29.026721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dfc8a-e439-4528-aa2e-caf69f2f670b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:29.028326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dfc8a-e439-4528-aa2e-caf69f2f670b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:29.040110 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 157/333] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Oct 14 17:42:28 2022] GET /identity/v3/role_assignments?group.id=2b357f7d938a4122998fc0f75fea508a&scope.project.id=8a5d8b8f16a64419ab27d85700e9bdb5&role.id=1cc2ea78f84c4491967839cfead3fd26 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:29.048613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7b725-0d73-4e02-bbbd-373cc65ebf34 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:29.090052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109dd99-9e01-4077-bc67-101b5e1c2383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:29.090741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109dd99-9e01-4077-bc67-101b5e1c2383 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:29.091020 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109dd99-9e01-4077-bc67-101b5e1c2383 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:29.091341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109dd99-9e01-4077-bc67-101b5e1c2383 None admin] PATH_INFO: `/v3/projects/a41cbe9ae3574988a4c391824f7f340a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:29.092634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109dd99-9e01-4077-bc67-101b5e1c2383 None admin] RBAC: Authorizing `identity:get_project(project_id=a41cbe9ae3574988a4c391824f7f340a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:29.094160 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109dd99-9e01-4077-bc67-101b5e1c2383 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:29.096171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 177/334] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:42:28 2022] GET /identity/v3/projects/a41cbe9ae3574988a4c391824f7f340a => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:29.102820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dfc8a-e439-4528-aa2e-caf69f2f670b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:29.156304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:29.218839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dfc8a-e439-4528-aa2e-caf69f2f670b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:29.254081 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f86bb-02ef-4aa1-9efa-fd5085e19b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:29.254715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f86bb-02ef-4aa1-9efa-fd5085e19b44 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:29.254991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f86bb-02ef-4aa1-9efa-fd5085e19b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:29.255264 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f86bb-02ef-4aa1-9efa-fd5085e19b44 None admin] PATH_INFO: `/v3/groups/bc1ea5c92b864a26bc18658925ce010b` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:29.257019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f86bb-02ef-4aa1-9efa-fd5085e19b44 None admin] RBAC: Authorizing `identity:get_group(group_id=bc1ea5c92b864a26bc18658925ce010b)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:29.258568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f86bb-02ef-4aa1-9efa-fd5085e19b44 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:29.263985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 158/335] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Oct 14 17:42:29 2022] GET /identity/v3/groups/bc1ea5c92b864a26bc18658925ce010b => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:29.266754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109dd99-9e01-4077-bc67-101b5e1c2383 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:29.369250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:29.372534 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109dd99-9e01-4077-bc67-101b5e1c2383 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:29.409368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72b334-093d-40c6-a1d0-bfba91381d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:29.409894 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72b334-093d-40c6-a1d0-bfba91381d41 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:29.410175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72b334-093d-40c6-a1d0-bfba91381d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:29.410439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72b334-093d-40c6-a1d0-bfba91381d41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:29.411156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72b334-093d-40c6-a1d0-bfba91381d41 None admin] RBAC: Adding query filter params (group.id=bc1ea5c92b864a26bc18658925ce010b, role.id=45b67ba9922c4b16baae3930d3de1b6c, scope.project.id=a41cbe9ae3574988a4c391824f7f340a) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:29.411460 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72b334-093d-40c6-a1d0-bfba91381d41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:29.413735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72b334-093d-40c6-a1d0-bfba91381d41 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:29.421384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 178/336] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Oct 14 17:42:29 2022] GET /identity/v3/role_assignments?group.id=bc1ea5c92b864a26bc18658925ce010b&scope.project.id=a41cbe9ae3574988a4c391824f7f340a&role.id=45b67ba9922c4b16baae3930d3de1b6c => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:32.285908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09311b53-2e5d-45b5-871d-372e162018c0 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:32.286963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09311b53-2e5d-45b5-871d-372e162018c0 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:32.287454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09311b53-2e5d-45b5-871d-372e162018c0 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:32.289203 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 159/337] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:32.294499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33a4e7-309f-40f7-99bf-688a4fc23c7c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:32.294928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33a4e7-309f-40f7-99bf-688a4fc23c7c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:32.295038 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33a4e7-309f-40f7-99bf-688a4fc23c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:32.322116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ce33a4e7-309f-40f7-99bf-688a4fc23c7c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:32.323611 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ce33a4e7-309f-40f7-99bf-688a4fc23c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:32.352152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b5df9-9973-4636-bb63-47ffb5b5ac1f None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:32.352662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b5df9-9973-4636-bb63-47ffb5b5ac1f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:32.353140 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b5df9-9973-4636-bb63-47ffb5b5ac1f None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:32.354706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 160/338] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:42:32.361195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b36586-ef0c-47b5-a493-db3156611f3d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:32.361712 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b36586-ef0c-47b5-a493-db3156611f3d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:32.362190 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b36586-ef0c-47b5-a493-db3156611f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:32.404943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c6b36586-ef0c-47b5-a493-db3156611f3d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:32.406165 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c6b36586-ef0c-47b5-a493-db3156611f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:32.433397 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 179/339] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:32.518450 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 161/340] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:32.614864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e641dcf-ed83-4e8f-bcf5-fbdcb394a129 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:32.615509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e641dcf-ed83-4e8f-bcf5-fbdcb394a129 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:32.616071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e641dcf-ed83-4e8f-bcf5-fbdcb394a129 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:32.617883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 180/341] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:32.623595 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f1a90-53ab-4245-bd05-108d71070a3c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:32.624160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f1a90-53ab-4245-bd05-108d71070a3c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:32.624644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f1a90-53ab-4245-bd05-108d71070a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:32.653444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a50f1a90-53ab-4245-bd05-108d71070a3c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:32.655062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a50f1a90-53ab-4245-bd05-108d71070a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:32.701202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:32.744418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 162/342] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:32.848218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e641dcf-ed83-4e8f-bcf5-fbdcb394a129 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:32.896494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f1a90-53ab-4245-bd05-108d71070a3c None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:32.991975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:33.007109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e641dcf-ed83-4e8f-bcf5-fbdcb394a129 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:33.010520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f1a90-53ab-4245-bd05-108d71070a3c None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:33.070898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e14450-9ca6-4122-865a-b7a827cac013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:33.071541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e14450-9ca6-4122-865a-b7a827cac013 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.071865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e14450-9ca6-4122-865a-b7a827cac013 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.072165 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e14450-9ca6-4122-865a-b7a827cac013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.089037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770d5b9-bd68-46e2-8154-506d43fafdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:33.089673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770d5b9-bd68-46e2-8154-506d43fafdc7 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.089946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770d5b9-bd68-46e2-8154-506d43fafdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.090211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770d5b9-bd68-46e2-8154-506d43fafdc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.091806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e14450-9ca6-4122-865a-b7a827cac013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:33.094571 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e14450-9ca6-4122-865a-b7a827cac013 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:33.112626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770d5b9-bd68-46e2-8154-506d43fafdc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:33.114155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770d5b9-bd68-46e2-8154-506d43fafdc7 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-d4e14450-9ca6-4122-865a-b7a827cac013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:33.129700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:33.130718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:33.132405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:33.132405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:33.132405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-d770d5b9-bd68-46e2-8154-506d43fafdc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:33.134208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:33.135062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:33.136327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:33.136327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:33.136327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:33.136327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 181/343] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:33.137408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 163/344] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:33.141935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770d5b9-bd68-46e2-8154-506d43fafdc7 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:33.143422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e14450-9ca6-4122-865a-b7a827cac013 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:33.170609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:33.247995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770d5b9-bd68-46e2-8154-506d43fafdc7 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:33.253112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e14450-9ca6-4122-865a-b7a827cac013 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:33.283112 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c84b-d697-4e81-b00c-1d7ff0ce6b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:33.284068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c84b-d697-4e81-b00c-1d7ff0ce6b57 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.284479 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c84b-d697-4e81-b00c-1d7ff0ce6b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.284902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c84b-d697-4e81-b00c-1d7ff0ce6b57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.288584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415541da-27e2-40be-b925-e1ccf52f96fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:33.289193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415541da-27e2-40be-b925-e1ccf52f96fa None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.289468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415541da-27e2-40be-b925-e1ccf52f96fa None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.289736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415541da-27e2-40be-b925-e1ccf52f96fa None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.290374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415541da-27e2-40be-b925-e1ccf52f96fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:33.290674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415541da-27e2-40be-b925-e1ccf52f96fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:33.292064 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415541da-27e2-40be-b925-e1ccf52f96fa None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:33.297167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9c84b-d697-4e81-b00c-1d7ff0ce6b57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:33.298598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9c84b-d697-4e81-b00c-1d7ff0ce6b57 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:33.305734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:33.314198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 164/345] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:33.314198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415541da-27e2-40be-b925-e1ccf52f96fa None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-d9f9c84b-d697-4e81-b00c-1d7ff0ce6b57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:33.314447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:33.315239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:33.320553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:33.320553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:33.320553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:33.320553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 182/346] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:33.320553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5df81-115b-4dca-bf1f-929580fa9627 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.320553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5df81-115b-4dca-bf1f-929580fa9627 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.320553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5df81-115b-4dca-bf1f-929580fa9627 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.320553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 183/347] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:33.320553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5df81-115b-4dca-bf1f-929580fa9627 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:33.416091 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:33.435763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415541da-27e2-40be-b925-e1ccf52f96fa None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:33.440043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5df81-115b-4dca-bf1f-929580fa9627 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:33.472370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b4db4-e90e-471a-9873-8cd4b0d59abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:33.472979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b4db4-e90e-471a-9873-8cd4b0d59abe None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.473255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b4db4-e90e-471a-9873-8cd4b0d59abe None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.473526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b4db4-e90e-471a-9873-8cd4b0d59abe None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.474188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b4db4-e90e-471a-9873-8cd4b0d59abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:33.474452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b4db4-e90e-471a-9873-8cd4b0d59abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:33.475344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd55d2-653c-470e-82db-e4ceb7f0c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:33.475898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b4db4-e90e-471a-9873-8cd4b0d59abe None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:33.476079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd55d2-653c-470e-82db-e4ceb7f0c8b7 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.476317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd55d2-653c-470e-82db-e4ceb7f0c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.476588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd55d2-653c-470e-82db-e4ceb7f0c8b7 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.477337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd55d2-653c-470e-82db-e4ceb7f0c8b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:33.478924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd55d2-653c-470e-82db-e4ceb7f0c8b7 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:33.491565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 165/348] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:33.491565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40d9c0-e7d2-4d05-90c9-46b97ca0fb95 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.491565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40d9c0-e7d2-4d05-90c9-46b97ca0fb95 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.491565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40d9c0-e7d2-4d05-90c9-46b97ca0fb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.506265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 184/349] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:42:33 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:42:33.507899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd55d2-653c-470e-82db-e4ceb7f0c8b7 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:33.508086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6d40d9c0-e7d2-4d05-90c9-46b97ca0fb95 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:33.509458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6d40d9c0-e7d2-4d05-90c9-46b97ca0fb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:33.569682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:33.593552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 166/350] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:33.594754 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40d9c0-e7d2-4d05-90c9-46b97ca0fb95 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:33.639384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd55d2-653c-470e-82db-e4ceb7f0c8b7 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:33.682096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:33.691015 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689e573-fcba-4be8-8d44-55593a48d695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:33.692211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689e573-fcba-4be8-8d44-55593a48d695 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.692650 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689e573-fcba-4be8-8d44-55593a48d695 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.693382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689e573-fcba-4be8-8d44-55593a48d695 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.695179 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689e573-fcba-4be8-8d44-55593a48d695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:33.695627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689e573-fcba-4be8-8d44-55593a48d695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:33.697182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689e573-fcba-4be8-8d44-55593a48d695 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:33.711407 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 185/351] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:33.712625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba19ef6-30f6-468a-9de3-1c7135f7d3aa None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.712906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba19ef6-30f6-468a-9de3-1c7135f7d3aa None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.713175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba19ef6-30f6-468a-9de3-1c7135f7d3aa None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.714623 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 186/352] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:33.717688 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba19ef6-30f6-468a-9de3-1c7135f7d3aa None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:33.727621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40d9c0-e7d2-4d05-90c9-46b97ca0fb95 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:33.776371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865ea96-b28a-4659-b2e9-1afb92d65fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:33.777329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865ea96-b28a-4659-b2e9-1afb92d65fd9 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.778037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865ea96-b28a-4659-b2e9-1afb92d65fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.778994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865ea96-b28a-4659-b2e9-1afb92d65fd9 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.780230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865ea96-b28a-4659-b2e9-1afb92d65fd9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:33.782139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865ea96-b28a-4659-b2e9-1afb92d65fd9 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:33.807980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 167/353] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:42:33 2022] POST /identity/v3/users => generated 246 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:42:33.809653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ec342-64e9-48d5-9879-bddc9224862f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.810251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ec342-64e9-48d5-9879-bddc9224862f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.810797 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ec342-64e9-48d5-9879-bddc9224862f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.837128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2b7ec342-64e9-48d5-9879-bddc9224862f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:33.839992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2b7ec342-64e9-48d5-9879-bddc9224862f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:33.846362 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba19ef6-30f6-468a-9de3-1c7135f7d3aa None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:33.863550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:33.887946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226384e6-5d56-4b45-b8a4-6c0f87e3a199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:33.888578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226384e6-5d56-4b45-b8a4-6c0f87e3a199 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:33.888857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226384e6-5d56-4b45-b8a4-6c0f87e3a199 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:33.889180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226384e6-5d56-4b45-b8a4-6c0f87e3a199 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:33.889851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226384e6-5d56-4b45-b8a4-6c0f87e3a199 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:33.891719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226384e6-5d56-4b45-b8a4-6c0f87e3a199 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:33.919339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 187/354] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:42:33 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:42:33.920344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226384e6-5d56-4b45-b8a4-6c0f87e3a199 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:33.934372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 168/355] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:34.044518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226384e6-5d56-4b45-b8a4-6c0f87e3a199 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:34.083734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e870cc-49c4-4a26-b826-d2783657386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:34.084412 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e870cc-49c4-4a26-b826-d2783657386b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:34.084727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e870cc-49c4-4a26-b826-d2783657386b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:34.084979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e870cc-49c4-4a26-b826-d2783657386b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:34.097616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e870cc-49c4-4a26-b826-d2783657386b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:34.099158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e870cc-49c4-4a26-b826-d2783657386b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-e5e870cc-49c4-4a26-b826-d2783657386b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:34.115391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:34.116499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:34.117481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:34.117481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:34.117481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:34.117481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 188/356] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:34.123020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ec342-64e9-48d5-9879-bddc9224862f None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:34.193117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:34.241963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ec342-64e9-48d5-9879-bddc9224862f None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:34.247302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e870cc-49c4-4a26-b826-d2783657386b None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:34.277509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:34.292968 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b6ba5-4c53-4513-92a8-ef16d11d8781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:34.293619 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b6ba5-4c53-4513-92a8-ef16d11d8781 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:34.293850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b6ba5-4c53-4513-92a8-ef16d11d8781 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:34.294118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b6ba5-4c53-4513-92a8-ef16d11d8781 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:34.294815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b6ba5-4c53-4513-92a8-ef16d11d8781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:34.295056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b6ba5-4c53-4513-92a8-ef16d11d8781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:34.296623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b6ba5-4c53-4513-92a8-ef16d11d8781 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:34.309942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 169/357] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:34.315307 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b6ba5-4c53-4513-92a8-ef16d11d8781 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:34.375284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e870cc-49c4-4a26-b826-d2783657386b None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:34.415485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ef6b3-51d4-490f-b0f6-206aa8291676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:34.416221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ef6b3-51d4-490f-b0f6-206aa8291676 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:34.416504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ef6b3-51d4-490f-b0f6-206aa8291676 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:34.416782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ef6b3-51d4-490f-b0f6-206aa8291676 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:34.417465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ef6b3-51d4-490f-b0f6-206aa8291676 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:34.419132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ef6b3-51d4-490f-b0f6-206aa8291676 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:34.421526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b6ba5-4c53-4513-92a8-ef16d11d8781 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:34.427081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.sql.core [None req-042ef6b3-51d4-490f-b0f6-206aa8291676 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 14 17:42:34.427081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 14 17:42:34.427081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [parameters: {'id': 'cd49be53d0e3487b82fa1109e92e2db1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 14 17:42:34.427081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81267) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-042ef6b3-51d4-490f-b0f6-206aa8291676 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application conn.query(q) Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result.read() Oct 14 17:42:34.435267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return ref.to_dict() Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 17:42:34.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application next(self.gen) Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application yield resource Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application next(self.gen) Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise self.value Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application session.commit() Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 17:42:34.437921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.session.flush() Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application compat.raise_( Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 17:42:34.440070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 17:42:34.446026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application conn.query(q) Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result.read() Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 17:42:34.447807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [parameters: {'id': 'cd49be53d0e3487b82fa1109e92e2db1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:34.448965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 14 17:42:34.450578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:34.452261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 189/358] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Oct 14 17:42:34 2022] POST /identity/v3/roles => generated 144 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:34.452261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ef6b3-51d4-490f-b0f6-206aa8291676 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:34.452508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:34.463111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd04b7d-ab38-4b10-99e1-68a67584aab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:34.463814 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04b7d-ab38-4b10-99e1-68a67584aab7 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:34.464191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04b7d-ab38-4b10-99e1-68a67584aab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:34.464526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04b7d-ab38-4b10-99e1-68a67584aab7 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:34.465258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd04b7d-ab38-4b10-99e1-68a67584aab7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:34.466759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd04b7d-ab38-4b10-99e1-68a67584aab7 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:34.492317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 170/359] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:42:34 2022] POST /identity/v3/users => generated 249 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:42:34.554759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:34.572719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ef6b3-51d4-490f-b0f6-206aa8291676 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:34.611769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667571a1-93df-4cb6-8fba-b1bc67d1217b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:34.612541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667571a1-93df-4cb6-8fba-b1bc67d1217b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:34.612820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667571a1-93df-4cb6-8fba-b1bc67d1217b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:34.613092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667571a1-93df-4cb6-8fba-b1bc67d1217b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:34.626105 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667571a1-93df-4cb6-8fba-b1bc67d1217b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-667571a1-93df-4cb6-8fba-b1bc67d1217b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:34.632769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:34.635097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:34.637376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 190/360] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:42:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:34.640663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd04b7d-ab38-4b10-99e1-68a67584aab7 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:34.791593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd04b7d-ab38-4b10-99e1-68a67584aab7 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:34.828304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424964b-438e-41d4-878f-e788d64209af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:34.828839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424964b-438e-41d4-878f-e788d64209af None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:34.829116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424964b-438e-41d4-878f-e788d64209af None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:34.829388 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424964b-438e-41d4-878f-e788d64209af None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:34.830057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424964b-438e-41d4-878f-e788d64209af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:34.830353 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424964b-438e-41d4-878f-e788d64209af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:34.831739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424964b-438e-41d4-878f-e788d64209af None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:34.840493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 171/361] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:36.228693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcc15e-7ee6-4d3d-8cc1-89574c1e59ee None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:36.228693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcc15e-7ee6-4d3d-8cc1-89574c1e59ee None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:36.228693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcc15e-7ee6-4d3d-8cc1-89574c1e59ee None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:36.229987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 191/362] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:36.235590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb9cd4-1fcd-4bd9-a68d-b2f350ec06f9 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:36.236093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb9cd4-1fcd-4bd9-a68d-b2f350ec06f9 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:36.236274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb9cd4-1fcd-4bd9-a68d-b2f350ec06f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:36.265518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a4eb9cd4-1fcd-4bd9-a68d-b2f350ec06f9 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:36.266699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a4eb9cd4-1fcd-4bd9-a68d-b2f350ec06f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:36.309594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:36.365500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 172/363] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:36.549618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088b73a-a37d-4fc8-bf6b-b818056c5c45 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:36.549618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088b73a-a37d-4fc8-bf6b-b818056c5c45 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:36.549618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088b73a-a37d-4fc8-bf6b-b818056c5c45 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:36.550492 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 192/364] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:36.555634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777df7d2-c430-4142-86a3-0e881d92ad93 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:36.555940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777df7d2-c430-4142-86a3-0e881d92ad93 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:36.556216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777df7d2-c430-4142-86a3-0e881d92ad93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:36.584193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-777df7d2-c430-4142-86a3-0e881d92ad93 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:36.586039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-777df7d2-c430-4142-86a3-0e881d92ad93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:36.675247 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 173/365] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:36.786659 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088b73a-a37d-4fc8-bf6b-b818056c5c45 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:36.901545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088b73a-a37d-4fc8-bf6b-b818056c5c45 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:36.936939 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aabb55-191a-4f10-9a61-4ff3231f5910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:36.937907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aabb55-191a-4f10-9a61-4ff3231f5910 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:36.938398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aabb55-191a-4f10-9a61-4ff3231f5910 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:36.938879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aabb55-191a-4f10-9a61-4ff3231f5910 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:36.958778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aabb55-191a-4f10-9a61-4ff3231f5910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-08aabb55-191a-4f10-9a61-4ff3231f5910 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:36.964955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:36.965859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:36.966647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:36.969073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 193/366] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:36.975002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777df7d2-c430-4142-86a3-0e881d92ad93 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:37.010205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc50de-21e9-42ba-9649-9ca6be7e5b8e None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.010810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc50de-21e9-42ba-9649-9ca6be7e5b8e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.011274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc50de-21e9-42ba-9649-9ca6be7e5b8e None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.013365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 194/367] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:42:37.019489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728070a-5aab-4bd5-9a3e-a9a57cfd1652 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.020041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728070a-5aab-4bd5-9a3e-a9a57cfd1652 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.020507 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728070a-5aab-4bd5-9a3e-a9a57cfd1652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.041615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:37.048392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9728070a-5aab-4bd5-9a3e-a9a57cfd1652 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:37.049710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9728070a-5aab-4bd5-9a3e-a9a57cfd1652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:37.119401 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:37.130026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777df7d2-c430-4142-86a3-0e881d92ad93 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:37.145987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 195/368] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:37.147304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728070a-5aab-4bd5-9a3e-a9a57cfd1652 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:37.210083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a2a2-9915-4909-9707-2bccd11fc409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:37.210670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a2a2-9915-4909-9707-2bccd11fc409 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.210955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a2a2-9915-4909-9707-2bccd11fc409 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.211222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a2a2-9915-4909-9707-2bccd11fc409 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.211903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0a2a2-9915-4909-9707-2bccd11fc409 None admin] RBAC: Adding query filter params (name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:37.212213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0a2a2-9915-4909-9707-2bccd11fc409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:37.213745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0a2a2-9915-4909-9707-2bccd11fc409 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:37.232796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 174/369] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:37.237878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a2a2-9915-4909-9707-2bccd11fc409 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:37.267408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728070a-5aab-4bd5-9a3e-a9a57cfd1652 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:37.311963 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bf326-e2f0-4f24-8eda-d9216be063bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:37.312964 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bf326-e2f0-4f24-8eda-d9216be063bb None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.313438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bf326-e2f0-4f24-8eda-d9216be063bb None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.314150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bf326-e2f0-4f24-8eda-d9216be063bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.317515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:37.330139 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3bf326-e2f0-4f24-8eda-d9216be063bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-ce3bf326-e2f0-4f24-8eda-d9216be063bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:37.337335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:37.338244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:37.339037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:37.341969 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 196/370] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:37.347425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bf326-e2f0-4f24-8eda-d9216be063bb None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:37.427388 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a2a2-9915-4909-9707-2bccd11fc409 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:37.464789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5f9e3-3d23-436a-896b-d736f370bcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:37.465740 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5f9e3-3d23-436a-896b-d736f370bcf5 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.466204 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5f9e3-3d23-436a-896b-d736f370bcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.466661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5f9e3-3d23-436a-896b-d736f370bcf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.478811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5f9e3-3d23-436a-896b-d736f370bcf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:37.480559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5f9e3-3d23-436a-896b-d736f370bcf5 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:37.488547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bf326-e2f0-4f24-8eda-d9216be063bb None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-83d5f9e3-3d23-436a-896b-d736f370bcf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:37.495850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:37.496784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:37.497600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:37.497600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:37.497600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:37.499723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 175/371] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:37.501291 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae5a3a-51d2-4991-8f17-1e2e4099025b None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.501760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae5a3a-51d2-4991-8f17-1e2e4099025b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.502216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae5a3a-51d2-4991-8f17-1e2e4099025b None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.503713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 176/372] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:37.509082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae5a3a-51d2-4991-8f17-1e2e4099025b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:37.525875 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ab517-8588-4882-8703-ea210e75ef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:37.526572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ab517-8588-4882-8703-ea210e75ef2e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.526759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:37.526947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ab517-8588-4882-8703-ea210e75ef2e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.527180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ab517-8588-4882-8703-ea210e75ef2e None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.527850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ab517-8588-4882-8703-ea210e75ef2e None admin] RBAC: Adding query filter params (name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:37.528341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ab517-8588-4882-8703-ea210e75ef2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:37.529780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ab517-8588-4882-8703-ea210e75ef2e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:37.538539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 197/373] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:37.540963 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ab517-8588-4882-8703-ea210e75ef2e None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:37.608612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:37.630430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae5a3a-51d2-4991-8f17-1e2e4099025b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:37.655213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ab517-8588-4882-8703-ea210e75ef2e None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:37.667372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a5821-3b11-4c92-863e-433966004b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:37.668064 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a5821-3b11-4c92-863e-433966004b44 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.668340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a5821-3b11-4c92-863e-433966004b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.668606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a5821-3b11-4c92-863e-433966004b44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.681018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a5821-3b11-4c92-863e-433966004b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-940a5821-3b11-4c92-863e-433966004b44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:37.686556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:37.687736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:37.688605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 177/374] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:37.689387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94cc75-567b-456f-ab28-0f636387972d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.689669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94cc75-567b-456f-ab28-0f636387972d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.689931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94cc75-567b-456f-ab28-0f636387972d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.700608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af5cf-7e33-4f5a-936c-41ba9bf14c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:37.701282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af5cf-7e33-4f5a-936c-41ba9bf14c4b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.701585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af5cf-7e33-4f5a-936c-41ba9bf14c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.701853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af5cf-7e33-4f5a-936c-41ba9bf14c4b None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.702504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16af5cf-7e33-4f5a-936c-41ba9bf14c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:37.702801 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16af5cf-7e33-4f5a-936c-41ba9bf14c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:37.704231 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16af5cf-7e33-4f5a-936c-41ba9bf14c4b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:37.710313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-dc94cc75-567b-456f-ab28-0f636387972d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:37.711508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-dc94cc75-567b-456f-ab28-0f636387972d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:37.717405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 198/375] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:37.718000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af5cf-7e33-4f5a-936c-41ba9bf14c4b None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:37.781131 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:37.788035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 178/376] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:37.789067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94cc75-567b-456f-ab28-0f636387972d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:37.826722 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af5cf-7e33-4f5a-936c-41ba9bf14c4b None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:37.845857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:37.863381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5d92-5206-4dcf-aa05-2a42ae043a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:37.864025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5d92-5206-4dcf-aa05-2a42ae043a3b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.864297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5d92-5206-4dcf-aa05-2a42ae043a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.864561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5d92-5206-4dcf-aa05-2a42ae043a3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.874629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b5d92-5206-4dcf-aa05-2a42ae043a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:37.876090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b5d92-5206-4dcf-aa05-2a42ae043a3b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-e94b5d92-5206-4dcf-aa05-2a42ae043a3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:37.891443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:37.892368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:37.893204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:37.893204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:37.893204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:37.893363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94cc75-567b-456f-ab28-0f636387972d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:37.898232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 199/377] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:37.898729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5d92-5206-4dcf-aa05-2a42ae043a3b None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:37.929379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc568982-2dcc-4a6d-a27b-667e32ff06c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:37.930294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc568982-2dcc-4a6d-a27b-667e32ff06c9 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.930825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc568982-2dcc-4a6d-a27b-667e32ff06c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.931098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc568982-2dcc-4a6d-a27b-667e32ff06c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.931750 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc568982-2dcc-4a6d-a27b-667e32ff06c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:37.932073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc568982-2dcc-4a6d-a27b-667e32ff06c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:37.933440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc568982-2dcc-4a6d-a27b-667e32ff06c9 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:37.941369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 179/378] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:37.942566 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94696af-25a2-43dd-8b2d-9bfc9dc4f213 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:37.942846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94696af-25a2-43dd-8b2d-9bfc9dc4f213 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:37.943137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94696af-25a2-43dd-8b2d-9bfc9dc4f213 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:37.945461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 180/379] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:37.950641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94696af-25a2-43dd-8b2d-9bfc9dc4f213 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.005199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5d92-5206-4dcf-aa05-2a42ae043a3b None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.020053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:38.039871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576d08f-0f85-40ab-9ffc-9d9f30c6c8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.040518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576d08f-0f85-40ab-9ffc-9d9f30c6c8ad None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.040799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576d08f-0f85-40ab-9ffc-9d9f30c6c8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.041069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576d08f-0f85-40ab-9ffc-9d9f30c6c8ad None admin] PATH_INFO: `/v3/users/glance` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.048609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576d08f-0f85-40ab-9ffc-9d9f30c6c8ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:38.050097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576d08f-0f85-40ab-9ffc-9d9f30c6c8ad None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-d576d08f-0f85-40ab-9ffc-9d9f30c6c8ad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:38.063727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:38.064911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 200/380] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:38.065849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576d08f-0f85-40ab-9ffc-9d9f30c6c8ad None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.068404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94696af-25a2-43dd-8b2d-9bfc9dc4f213 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.095935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:38.104023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffea03-c1b6-4eaa-a56a-6c060d5db81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.104624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffea03-c1b6-4eaa-a56a-6c060d5db81f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.104919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffea03-c1b6-4eaa-a56a-6c060d5db81f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.105214 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffea03-c1b6-4eaa-a56a-6c060d5db81f None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.105870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffea03-c1b6-4eaa-a56a-6c060d5db81f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:38.106188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffea03-c1b6-4eaa-a56a-6c060d5db81f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:38.107667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffea03-c1b6-4eaa-a56a-6c060d5db81f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:38.119958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 181/381] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:38.121147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb72398-f5e2-42f6-ab83-d2ab2e26dfdc None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.121427 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb72398-f5e2-42f6-ab83-d2ab2e26dfdc None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.121890 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb72398-f5e2-42f6-ab83-d2ab2e26dfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.139821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2fb72398-f5e2-42f6-ab83-d2ab2e26dfdc None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:38.141021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2fb72398-f5e2-42f6-ab83-d2ab2e26dfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:38.175517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576d08f-0f85-40ab-9ffc-9d9f30c6c8ad None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.206839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 182/382] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:38.207675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb72398-f5e2-42f6-ab83-d2ab2e26dfdc None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.220234 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d64a73-04bd-4516-86a5-ae7cec309dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.220856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d64a73-04bd-4516-86a5-ae7cec309dfb None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.221131 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d64a73-04bd-4516-86a5-ae7cec309dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.221399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d64a73-04bd-4516-86a5-ae7cec309dfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.234913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d64a73-04bd-4516-86a5-ae7cec309dfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:38.236565 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d64a73-04bd-4516-86a5-ae7cec309dfb None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-62d64a73-04bd-4516-86a5-ae7cec309dfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:38.252706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:38.253673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:38.254516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:38.254516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:38.254516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:38.254516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 201/383] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:38.254516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d64a73-04bd-4516-86a5-ae7cec309dfb None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.269443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:38.321408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb72398-f5e2-42f6-ab83-d2ab2e26dfdc None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.340827 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:38.361011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d64a73-04bd-4516-86a5-ae7cec309dfb None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.361785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71baf5-6043-4874-ba29-e6dc6ecdccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.362419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71baf5-6043-4874-ba29-e6dc6ecdccf0 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.362693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71baf5-6043-4874-ba29-e6dc6ecdccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.362958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71baf5-6043-4874-ba29-e6dc6ecdccf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.372986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71baf5-6043-4874-ba29-e6dc6ecdccf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-0b71baf5-6043-4874-ba29-e6dc6ecdccf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:38.377760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:38.378650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:38.379446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 183/384] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:38.380158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71baf5-6043-4874-ba29-e6dc6ecdccf0 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.406078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99b16c-8452-4377-8b87-e355f48f6221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.407024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99b16c-8452-4377-8b87-e355f48f6221 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.407448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99b16c-8452-4377-8b87-e355f48f6221 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.407909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99b16c-8452-4377-8b87-e355f48f6221 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.408924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99b16c-8452-4377-8b87-e355f48f6221 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:38.409493 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99b16c-8452-4377-8b87-e355f48f6221 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:38.410929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99b16c-8452-4377-8b87-e355f48f6221 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:38.454718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 202/385] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:38.454718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99b16c-8452-4377-8b87-e355f48f6221 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.487739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71baf5-6043-4874-ba29-e6dc6ecdccf0 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.500717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:38.523078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7db76-72bc-4a90-be32-04f9a6768941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.523396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7db76-72bc-4a90-be32-04f9a6768941 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.523666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7db76-72bc-4a90-be32-04f9a6768941 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.523959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7db76-72bc-4a90-be32-04f9a6768941 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.531176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7db76-72bc-4a90-be32-04f9a6768941 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:38.532661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7db76-72bc-4a90-be32-04f9a6768941 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-15d7db76-72bc-4a90-be32-04f9a6768941 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:38.545316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:38.546195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:38.547097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 184/386] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:38.547652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7db76-72bc-4a90-be32-04f9a6768941 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.552567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99b16c-8452-4377-8b87-e355f48f6221 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.587474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5b6ff-07ed-4e93-bb8b-ce563904dd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.588186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5b6ff-07ed-4e93-bb8b-ce563904dd51 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.588524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5b6ff-07ed-4e93-bb8b-ce563904dd51 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.588855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5b6ff-07ed-4e93-bb8b-ce563904dd51 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.589581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5b6ff-07ed-4e93-bb8b-ce563904dd51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:38.589887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5b6ff-07ed-4e93-bb8b-ce563904dd51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:38.591272 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5b6ff-07ed-4e93-bb8b-ce563904dd51 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:38.599239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:38.603524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 203/387] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:38.604526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5b6ff-07ed-4e93-bb8b-ce563904dd51 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.650575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7db76-72bc-4a90-be32-04f9a6768941 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.685230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd197e09-2ca5-432b-a648-618e1f0deb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.685854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd197e09-2ca5-432b-a648-618e1f0deb51 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.686127 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd197e09-2ca5-432b-a648-618e1f0deb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.686393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd197e09-2ca5-432b-a648-618e1f0deb51 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.687044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd197e09-2ca5-432b-a648-618e1f0deb51 None admin] RBAC: Adding query filter params (name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:38.687336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd197e09-2ca5-432b-a648-618e1f0deb51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:38.688742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd197e09-2ca5-432b-a648-618e1f0deb51 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:38.697384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 185/388] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:38.698153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd197e09-2ca5-432b-a648-618e1f0deb51 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.702740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5b6ff-07ed-4e93-bb8b-ce563904dd51 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.731864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:38.736725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b399bd4-37da-4d06-9ba8-a714d390d359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.737346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b399bd4-37da-4d06-9ba8-a714d390d359 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.737618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b399bd4-37da-4d06-9ba8-a714d390d359 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.737886 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b399bd4-37da-4d06-9ba8-a714d390d359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.748522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b399bd4-37da-4d06-9ba8-a714d390d359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:38.749978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b399bd4-37da-4d06-9ba8-a714d390d359 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-0b399bd4-37da-4d06-9ba8-a714d390d359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:38.764629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:38.765703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:38.767157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:38.767157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:38.767157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:38.767157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 204/389] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:38.767157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b399bd4-37da-4d06-9ba8-a714d390d359 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.800157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd197e09-2ca5-432b-a648-618e1f0deb51 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.819738 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:38.833571 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ae9d8-1801-40ff-90e7-3e0adafbc02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.834219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ae9d8-1801-40ff-90e7-3e0adafbc02e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.835142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ae9d8-1801-40ff-90e7-3e0adafbc02e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.835428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ae9d8-1801-40ff-90e7-3e0adafbc02e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.846060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ae9d8-1801-40ff-90e7-3e0adafbc02e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:38.847500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ae9d8-1801-40ff-90e7-3e0adafbc02e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-027ae9d8-1801-40ff-90e7-3e0adafbc02e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:38.861779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:38.862745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:38.863593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:38.863593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:38.863593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:38.863593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 186/390] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:38.863593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ae9d8-1801-40ff-90e7-3e0adafbc02e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.870573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b399bd4-37da-4d06-9ba8-a714d390d359 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:38.903803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705329bd-9f27-47f7-9005-5e323931a97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:38.904421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705329bd-9f27-47f7-9005-5e323931a97f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:38.904694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705329bd-9f27-47f7-9005-5e323931a97f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:38.904959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705329bd-9f27-47f7-9005-5e323931a97f None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:38.905678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705329bd-9f27-47f7-9005-5e323931a97f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:38.905974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705329bd-9f27-47f7-9005-5e323931a97f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:38.907408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705329bd-9f27-47f7-9005-5e323931a97f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:38.925300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 205/391] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:38.926223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705329bd-9f27-47f7-9005-5e323931a97f None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:38.957325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:38.972493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ae9d8-1801-40ff-90e7-3e0adafbc02e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.005586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19b6ab-8eb2-433d-914a-7f7380f33288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.006211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19b6ab-8eb2-433d-914a-7f7380f33288 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.007194 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19b6ab-8eb2-433d-914a-7f7380f33288 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.007466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19b6ab-8eb2-433d-914a-7f7380f33288 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.015487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19b6ab-8eb2-433d-914a-7f7380f33288 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.016940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19b6ab-8eb2-433d-914a-7f7380f33288 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.024334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705329bd-9f27-47f7-9005-5e323931a97f None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-9c19b6ab-8eb2-433d-914a-7f7380f33288 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:39.028208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:39.029134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 187/392] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:39.030134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19b6ab-8eb2-433d-914a-7f7380f33288 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.035690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:39.059581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741f7ba-83e6-4a94-9c8a-42be3273d471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.060173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741f7ba-83e6-4a94-9c8a-42be3273d471 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.060447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741f7ba-83e6-4a94-9c8a-42be3273d471 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.060714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741f7ba-83e6-4a94-9c8a-42be3273d471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.071411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741f7ba-83e6-4a94-9c8a-42be3273d471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.072892 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741f7ba-83e6-4a94-9c8a-42be3273d471 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-a741f7ba-83e6-4a94-9c8a-42be3273d471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:39.087185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:39.088230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:39.089065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:39.089065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:39.089065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:39.089065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 206/393] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:39.089065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741f7ba-83e6-4a94-9c8a-42be3273d471 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.133091 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19b6ab-8eb2-433d-914a-7f7380f33288 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.165748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa1d19-cb16-4daf-98ae-f893abb3518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.166351 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa1d19-cb16-4daf-98ae-f893abb3518b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.166624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa1d19-cb16-4daf-98ae-f893abb3518b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.166892 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa1d19-cb16-4daf-98ae-f893abb3518b None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.167528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa1d19-cb16-4daf-98ae-f893abb3518b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:39.167829 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa1d19-cb16-4daf-98ae-f893abb3518b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.169744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa1d19-cb16-4daf-98ae-f893abb3518b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.181788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 188/394] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:39.182576 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa1d19-cb16-4daf-98ae-f893abb3518b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.187614 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741f7ba-83e6-4a94-9c8a-42be3273d471 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.192135 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:39.221406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57708d36-326d-4eb5-8460-f8e8fbd617e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.222005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57708d36-326d-4eb5-8460-f8e8fbd617e8 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.222280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57708d36-326d-4eb5-8460-f8e8fbd617e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.222546 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57708d36-326d-4eb5-8460-f8e8fbd617e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.223222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57708d36-326d-4eb5-8460-f8e8fbd617e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:39.223529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57708d36-326d-4eb5-8460-f8e8fbd617e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.224923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57708d36-326d-4eb5-8460-f8e8fbd617e8 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.237175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 207/395] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:39.237948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57708d36-326d-4eb5-8460-f8e8fbd617e8 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.264507 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:39.282876 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa1d19-cb16-4daf-98ae-f893abb3518b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.316726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090fc6b-7a0a-41e6-ae2c-e7a9953f56d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.317278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090fc6b-7a0a-41e6-ae2c-e7a9953f56d8 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.317559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090fc6b-7a0a-41e6-ae2c-e7a9953f56d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.317830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090fc6b-7a0a-41e6-ae2c-e7a9953f56d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.328124 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090fc6b-7a0a-41e6-ae2c-e7a9953f56d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.329549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090fc6b-7a0a-41e6-ae2c-e7a9953f56d8 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.337910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57708d36-326d-4eb5-8460-f8e8fbd617e8 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-c090fc6b-7a0a-41e6-ae2c-e7a9953f56d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:39.343340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:39.344232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:39.345102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:39.345102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:39.345102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:39.345102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 189/396] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:39.345331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090fc6b-7a0a-41e6-ae2c-e7a9953f56d8 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.375212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe34cb-e3f8-4cd4-8592-d1e27a9c8183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.375814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe34cb-e3f8-4cd4-8592-d1e27a9c8183 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.376124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe34cb-e3f8-4cd4-8592-d1e27a9c8183 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.376391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe34cb-e3f8-4cd4-8592-d1e27a9c8183 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.377106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe34cb-e3f8-4cd4-8592-d1e27a9c8183 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:39.377404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe34cb-e3f8-4cd4-8592-d1e27a9c8183 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.378838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe34cb-e3f8-4cd4-8592-d1e27a9c8183 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.396268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 208/397] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:39.397247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe34cb-e3f8-4cd4-8592-d1e27a9c8183 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.412500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:39.456033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090fc6b-7a0a-41e6-ae2c-e7a9953f56d8 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.477720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:39.489220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26684326-4e58-4b5f-bd6b-9bcfc6c5a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.489814 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26684326-4e58-4b5f-bd6b-9bcfc6c5a04e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.490086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26684326-4e58-4b5f-bd6b-9bcfc6c5a04e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.490356 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26684326-4e58-4b5f-bd6b-9bcfc6c5a04e None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.491624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26684326-4e58-4b5f-bd6b-9bcfc6c5a04e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:39.491948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26684326-4e58-4b5f-bd6b-9bcfc6c5a04e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.493308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26684326-4e58-4b5f-bd6b-9bcfc6c5a04e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.497574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe34cb-e3f8-4cd4-8592-d1e27a9c8183 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.504702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 190/398] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:39.505467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26684326-4e58-4b5f-bd6b-9bcfc6c5a04e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.532393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea780821-34e1-427a-a599-39df92c2998a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.533041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea780821-34e1-427a-a599-39df92c2998a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.533317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea780821-34e1-427a-a599-39df92c2998a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.533585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea780821-34e1-427a-a599-39df92c2998a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.540571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea780821-34e1-427a-a599-39df92c2998a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.542026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea780821-34e1-427a-a599-39df92c2998a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-ea780821-34e1-427a-a599-39df92c2998a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:39.556083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:39.556975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:39.557828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:39.557828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:39.557828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:39.557828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:39.557828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:39.557828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:39.557828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:39.557828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:39.557828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 209/399] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:39.558289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea780821-34e1-427a-a599-39df92c2998a None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.609551 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26684326-4e58-4b5f-bd6b-9bcfc6c5a04e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.625980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:39.644496 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d049b3e-94fa-455b-afd2-4b35e3056c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.645122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d049b3e-94fa-455b-afd2-4b35e3056c0a None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.645386 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d049b3e-94fa-455b-afd2-4b35e3056c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.645654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d049b3e-94fa-455b-afd2-4b35e3056c0a None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.646287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d049b3e-94fa-455b-afd2-4b35e3056c0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.648496 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d049b3e-94fa-455b-afd2-4b35e3056c0a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.654539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea780821-34e1-427a-a599-39df92c2998a None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.670128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 191/400] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:42:39 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:42:39.670925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d049b3e-94fa-455b-afd2-4b35e3056c0a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.706608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0a4e6-99f6-4f86-8583-a54b142f2889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.707344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0a4e6-99f6-4f86-8583-a54b142f2889 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.707625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0a4e6-99f6-4f86-8583-a54b142f2889 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.707915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0a4e6-99f6-4f86-8583-a54b142f2889 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.708576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0a4e6-99f6-4f86-8583-a54b142f2889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:39.708877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0a4e6-99f6-4f86-8583-a54b142f2889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.710655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0a4e6-99f6-4f86-8583-a54b142f2889 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.719732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:39.724881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 210/401] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:39.726087 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0a4e6-99f6-4f86-8583-a54b142f2889 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.789709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d049b3e-94fa-455b-afd2-4b35e3056c0a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.828905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fa203-4560-441b-b5da-de97c8273d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.829641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fa203-4560-441b-b5da-de97c8273d3f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.829919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fa203-4560-441b-b5da-de97c8273d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.830187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fa203-4560-441b-b5da-de97c8273d3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.842516 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fa203-4560-441b-b5da-de97c8273d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.844024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fa203-4560-441b-b5da-de97c8273d3f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.852777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0a4e6-99f6-4f86-8583-a54b142f2889 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-827fa203-4560-441b-b5da-de97c8273d3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:39.860166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:39.861052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:39.861881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:39.861881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:39.861881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:39.861881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 192/402] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:39.863026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fa203-4560-441b-b5da-de97c8273d3f None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.897290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf9ff4-0722-4fdd-996b-6f899df45bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:39.898011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf9ff4-0722-4fdd-996b-6f899df45bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:39.898292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf9ff4-0722-4fdd-996b-6f899df45bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:39.898558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf9ff4-0722-4fdd-996b-6f899df45bb8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:39.906186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf9ff4-0722-4fdd-996b-6f899df45bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:39.907636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf9ff4-0722-4fdd-996b-6f899df45bb8 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:39.908914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-28cf9ff4-0722-4fdd-996b-6f899df45bb8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:39.919089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:39.920246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:39.921361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:39.921361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:39.921361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:39.921361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:39.921361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:39.921361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:39.921361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:39.921361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:39.921361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 17:42:39.921361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:39.924840 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 211/403] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:39.925657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf9ff4-0722-4fdd-996b-6f899df45bb8 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:39.982874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fa203-4560-441b-b5da-de97c8273d3f None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:39.985297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:40.027942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adabe67-b5f9-4764-99c6-f52adb727790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:40.028718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adabe67-b5f9-4764-99c6-f52adb727790 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:40.028991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adabe67-b5f9-4764-99c6-f52adb727790 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:40.029258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adabe67-b5f9-4764-99c6-f52adb727790 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:40.029912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf9ff4-0722-4fdd-996b-6f899df45bb8 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:40.030028 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adabe67-b5f9-4764-99c6-f52adb727790 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:40.030224 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adabe67-b5f9-4764-99c6-f52adb727790 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:40.031662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adabe67-b5f9-4764-99c6-f52adb727790 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:40.050742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 193/404] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:40.051562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adabe67-b5f9-4764-99c6-f52adb727790 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:40.071705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94d443-5e23-4788-97ef-3bd6594d76c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:40.072583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94d443-5e23-4788-97ef-3bd6594d76c0 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:40.073034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94d443-5e23-4788-97ef-3bd6594d76c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:40.073513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94d443-5e23-4788-97ef-3bd6594d76c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:40.080522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94d443-5e23-4788-97ef-3bd6594d76c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:40.082147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94d443-5e23-4788-97ef-3bd6594d76c0 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-0f94d443-5e23-4788-97ef-3bd6594d76c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:40.094248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:40.095274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:40.096328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:40.096328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:40.096328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:40.096328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:40.096328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:40.096328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:40.096328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:40.096328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:40.102726 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 212/405] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:40.103736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94d443-5e23-4788-97ef-3bd6594d76c0 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:40.149248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:40.168855 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adabe67-b5f9-4764-99c6-f52adb727790 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:40.211493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea453f-02c2-4970-97ee-42d792c050fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:40.212173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94d443-5e23-4788-97ef-3bd6594d76c0 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:40.212374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea453f-02c2-4970-97ee-42d792c050fe None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:40.212743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea453f-02c2-4970-97ee-42d792c050fe None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:40.213108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea453f-02c2-4970-97ee-42d792c050fe None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:40.213845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea453f-02c2-4970-97ee-42d792c050fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:40.214236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea453f-02c2-4970-97ee-42d792c050fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:40.215711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea453f-02c2-4970-97ee-42d792c050fe None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:40.228272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 194/406] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:40.229031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea453f-02c2-4970-97ee-42d792c050fe None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:40.237215 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:40.253778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6781f-956e-4e74-ae54-c545bae7e4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:40.254371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6781f-956e-4e74-ae54-c545bae7e4cb None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:40.254644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6781f-956e-4e74-ae54-c545bae7e4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:40.254910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6781f-956e-4e74-ae54-c545bae7e4cb None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:40.255634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6781f-956e-4e74-ae54-c545bae7e4cb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:40.255959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6781f-956e-4e74-ae54-c545bae7e4cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:40.258444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6781f-956e-4e74-ae54-c545bae7e4cb None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:40.276823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 213/407] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Oct 14 17:42:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:40.277469 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6781f-956e-4e74-ae54-c545bae7e4cb None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:40.341758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea453f-02c2-4970-97ee-42d792c050fe None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:40.382425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d5519-63e0-427a-bd70-e69e87461135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:40.383241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d5519-63e0-427a-bd70-e69e87461135 None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:40.383616 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d5519-63e0-427a-bd70-e69e87461135 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:40.383993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d5519-63e0-427a-bd70-e69e87461135 None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/ae3566d85c9c4c559ba672269ab38f09/roles/e1a3bea83ac349acb6d65b6afb5441d6` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:40.386862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6781f-956e-4e74-ae54-c545bae7e4cb None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:40.402478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d5519-63e0-427a-bd70-e69e87461135 None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=ae3566d85c9c4c559ba672269ab38f09, role_id=e1a3bea83ac349acb6d65b6afb5441d6)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:40.403854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d5519-63e0-427a-bd70-e69e87461135 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:40.407654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:40.421440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 195/408] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:40 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/ae3566d85c9c4c559ba672269ab38f09/roles/e1a3bea83ac349acb6d65b6afb5441d6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:40.422361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d5519-63e0-427a-bd70-e69e87461135 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:40.443735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36108f2f-d0df-476f-aa42-d9b41b092982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:40.445379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36108f2f-d0df-476f-aa42-d9b41b092982 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:40.445379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36108f2f-d0df-476f-aa42-d9b41b092982 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:40.445798 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36108f2f-d0df-476f-aa42-d9b41b092982 None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:40.446609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36108f2f-d0df-476f-aa42-d9b41b092982 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:40.446957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36108f2f-d0df-476f-aa42-d9b41b092982 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:40.448492 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36108f2f-d0df-476f-aa42-d9b41b092982 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:40.469221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 214/409] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:40.475145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36108f2f-d0df-476f-aa42-d9b41b092982 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:40.525714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:40.556766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d5519-63e0-427a-bd70-e69e87461135 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:40.603249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36108f2f-d0df-476f-aa42-d9b41b092982 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:40.605146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7112c7-80cc-4cda-917b-bf2faa829477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:40.606111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7112c7-80cc-4cda-917b-bf2faa829477 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:40.606535 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7112c7-80cc-4cda-917b-bf2faa829477 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:40.606948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7112c7-80cc-4cda-917b-bf2faa829477 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:40.615198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7112c7-80cc-4cda-917b-bf2faa829477 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:40.617265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7112c7-80cc-4cda-917b-bf2faa829477 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-fc7112c7-80cc-4cda-917b-bf2faa829477 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:40.632964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:40.634002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:40.635060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:40.635060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:40.635060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:40.635060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:40.635060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:40.635060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:40.635060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:40.635060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:40.637252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 196/410] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:40.638805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7112c7-80cc-4cda-917b-bf2faa829477 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:40.641979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801c722-d339-4204-aa64-a08a1505a70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:40.642608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801c722-d339-4204-aa64-a08a1505a70c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:40.642885 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801c722-d339-4204-aa64-a08a1505a70c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:40.643140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801c722-d339-4204-aa64-a08a1505a70c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:40.656351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801c722-d339-4204-aa64-a08a1505a70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:40.657721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801c722-d339-4204-aa64-a08a1505a70c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-2801c722-d339-4204-aa64-a08a1505a70c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:40.675116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:40.676226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:40.677073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:40.677073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:40.677073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:40.677073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 215/411] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:40.677589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801c722-d339-4204-aa64-a08a1505a70c None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:40.719906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:40.765745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7112c7-80cc-4cda-917b-bf2faa829477 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:40.794223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801c722-d339-4204-aa64-a08a1505a70c None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:40.797034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:40.805838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b9a0a-bab7-425f-9537-e2abd9acf0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:40.806500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9a0a-bab7-425f-9537-e2abd9acf0ff None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:40.806778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9a0a-bab7-425f-9537-e2abd9acf0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:40.807048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9a0a-bab7-425f-9537-e2abd9acf0ff None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/e42968016de8470c933655e921fb3394/roles/e1a3bea83ac349acb6d65b6afb5441d6` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:40.823648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b9a0a-bab7-425f-9537-e2abd9acf0ff None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=e42968016de8470c933655e921fb3394, role_id=e1a3bea83ac349acb6d65b6afb5441d6)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:40.825212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b9a0a-bab7-425f-9537-e2abd9acf0ff None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:40.841318 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4318ea-9978-4ad0-8d6f-4c6b5f8d1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:40.841870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 197/412] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:40 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/e42968016de8470c933655e921fb3394/roles/e1a3bea83ac349acb6d65b6afb5441d6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:40.842002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4318ea-9978-4ad0-8d6f-4c6b5f8d1a9c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:40.842230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4318ea-9978-4ad0-8d6f-4c6b5f8d1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:40.842687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b9a0a-bab7-425f-9537-e2abd9acf0ff None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:40.848002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4318ea-9978-4ad0-8d6f-4c6b5f8d1a9c None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:40.849837 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4318ea-9978-4ad0-8d6f-4c6b5f8d1a9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:40.850181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4318ea-9978-4ad0-8d6f-4c6b5f8d1a9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:40.856208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4318ea-9978-4ad0-8d6f-4c6b5f8d1a9c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:40.891110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 216/413] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:40.895455 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4318ea-9978-4ad0-8d6f-4c6b5f8d1a9c None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:41.038410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b9a0a-bab7-425f-9537-e2abd9acf0ff None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:41.046728 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:41.079528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf609489-6783-410b-b97d-ae9562e04c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:41.080165 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf609489-6783-410b-b97d-ae9562e04c23 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:41.080442 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf609489-6783-410b-b97d-ae9562e04c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:41.080713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf609489-6783-410b-b97d-ae9562e04c23 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:41.081401 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf609489-6783-410b-b97d-ae9562e04c23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:41.081702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf609489-6783-410b-b97d-ae9562e04c23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:41.083062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf609489-6783-410b-b97d-ae9562e04c23 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:41.101037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 198/414] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:41.107270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf609489-6783-410b-b97d-ae9562e04c23 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:41.123682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4318ea-9978-4ad0-8d6f-4c6b5f8d1a9c None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:41.177912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdf6b6-678f-471e-8970-c1291def0e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:41.179313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdf6b6-678f-471e-8970-c1291def0e40 None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:41.179313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdf6b6-678f-471e-8970-c1291def0e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:41.179596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdf6b6-678f-471e-8970-c1291def0e40 None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/8f51ee9e45fb4dda8d5b9740aa0ac828/roles/e1a3bea83ac349acb6d65b6afb5441d6` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:41.208991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdf6b6-678f-471e-8970-c1291def0e40 None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=8f51ee9e45fb4dda8d5b9740aa0ac828, role_id=e1a3bea83ac349acb6d65b6afb5441d6)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:41.211450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdf6b6-678f-471e-8970-c1291def0e40 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:41.229597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:41.238200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 217/415] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:40 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/8f51ee9e45fb4dda8d5b9740aa0ac828/roles/e1a3bea83ac349acb6d65b6afb5441d6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:41.263869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf609489-6783-410b-b97d-ae9562e04c23 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:41.313267 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29679fc9-43dd-429a-8c9d-ff7374cab4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:41.314302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679fc9-43dd-429a-8c9d-ff7374cab4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:41.314772 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679fc9-43dd-429a-8c9d-ff7374cab4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:41.315284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679fc9-43dd-429a-8c9d-ff7374cab4c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:41.323986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29679fc9-43dd-429a-8c9d-ff7374cab4c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:41.326209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29679fc9-43dd-429a-8c9d-ff7374cab4c6 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-29679fc9-43dd-429a-8c9d-ff7374cab4c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:41.339997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:41.340888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:41.341766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:41.341766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:41.341766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:41.341766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:41.341766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:41.341766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:41.341766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:41.341766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:41.345292 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 199/416] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:41.351198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdf6b6-678f-471e-8970-c1291def0e40 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:41.494901 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdf6b6-678f-471e-8970-c1291def0e40 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:41.540626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82313779-ddd3-40f0-9344-47bf805923e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:41.541610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313779-ddd3-40f0-9344-47bf805923e6 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:41.542082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313779-ddd3-40f0-9344-47bf805923e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:41.542542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313779-ddd3-40f0-9344-47bf805923e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:41.543424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82313779-ddd3-40f0-9344-47bf805923e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:41.543959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82313779-ddd3-40f0-9344-47bf805923e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:41.545597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82313779-ddd3-40f0-9344-47bf805923e6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:41.558654 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 218/417] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:41.564112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29679fc9-43dd-429a-8c9d-ff7374cab4c6 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:41.597404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:41.688700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29679fc9-43dd-429a-8c9d-ff7374cab4c6 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:41.733479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5e714-b892-40f3-a120-ae341ee201ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:41.734149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5e714-b892-40f3-a120-ae341ee201ad None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:41.734421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5e714-b892-40f3-a120-ae341ee201ad None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:41.734688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5e714-b892-40f3-a120-ae341ee201ad None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/3c17f274e4c14f2f99650844642437a2/roles/e1a3bea83ac349acb6d65b6afb5441d6` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:41.754502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5e714-b892-40f3-a120-ae341ee201ad None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=3c17f274e4c14f2f99650844642437a2, role_id=e1a3bea83ac349acb6d65b6afb5441d6)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:41.755974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5e714-b892-40f3-a120-ae341ee201ad None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:41.783488 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 200/418] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:41 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/3c17f274e4c14f2f99650844642437a2/roles/e1a3bea83ac349acb6d65b6afb5441d6 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:42.075349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5bdda-8a8e-43a3-845b-15486381b12b None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:42.075718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5bdda-8a8e-43a3-845b-15486381b12b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:42.075970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5bdda-8a8e-43a3-845b-15486381b12b None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:42.077442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 219/419] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:42.082857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b0542-e3aa-45bf-8c8c-5f10b5b2a929 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:42.083144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b0542-e3aa-45bf-8c8c-5f10b5b2a929 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:42.083417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b0542-e3aa-45bf-8c8c-5f10b5b2a929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:42.101634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-8a5b0542-e3aa-45bf-8c8c-5f10b5b2a929 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:42.102858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8a5b0542-e3aa-45bf-8c8c-5f10b5b2a929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:42.187310 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 201/420] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:42.569415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5bdda-8a8e-43a3-845b-15486381b12b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:42.639092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:42.693808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5bdda-8a8e-43a3-845b-15486381b12b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:42.743098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851ed1c-d539-4f73-8ced-871d68770ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:42.743731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851ed1c-d539-4f73-8ced-871d68770ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:42.744034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851ed1c-d539-4f73-8ced-871d68770ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:42.744306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851ed1c-d539-4f73-8ced-871d68770ce9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:42.764713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851ed1c-d539-4f73-8ced-871d68770ce9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-d851ed1c-d539-4f73-8ced-871d68770ce9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:42.769966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:42.770857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:42.772490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 220/421] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:42.779150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b0542-e3aa-45bf-8c8c-5f10b5b2a929 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:42.791529 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:42.946461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b0542-e3aa-45bf-8c8c-5f10b5b2a929 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:42.997438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f7eed-6493-47dc-a936-edab304968c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:42.998048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f7eed-6493-47dc-a936-edab304968c8 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:42.998323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f7eed-6493-47dc-a936-edab304968c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:42.998592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f7eed-6493-47dc-a936-edab304968c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:42.999257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f7eed-6493-47dc-a936-edab304968c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:42.999553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f7eed-6493-47dc-a936-edab304968c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:43.001301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f7eed-6493-47dc-a936-edab304968c8 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:43.001549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb968a-a9a5-4ad8-be61-bed719318cc8 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:43.001830 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb968a-a9a5-4ad8-be61-bed719318cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:43.002199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb968a-a9a5-4ad8-be61-bed719318cc8 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:43.003771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 221/422] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:43.009179 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06e48c-2fc4-4a1c-a094-569b8f7184bf None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:43.009500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06e48c-2fc4-4a1c-a094-569b8f7184bf None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:43.009781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06e48c-2fc4-4a1c-a094-569b8f7184bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:43.013207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 202/423] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:43.019067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f7eed-6493-47dc-a936-edab304968c8 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:43.039692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ff06e48c-2fc4-4a1c-a094-569b8f7184bf None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:43.042051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ff06e48c-2fc4-4a1c-a094-569b8f7184bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:43.135823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:43.175320 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 222/424] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:43.181299 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f7eed-6493-47dc-a936-edab304968c8 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:43.249164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c727cd-a499-4a5b-9f14-68f81d45c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:43.249709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c727cd-a499-4a5b-9f14-68f81d45c541 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:43.249979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c727cd-a499-4a5b-9f14-68f81d45c541 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:43.250242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c727cd-a499-4a5b-9f14-68f81d45c541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:43.264794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c727cd-a499-4a5b-9f14-68f81d45c541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:43.266220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c727cd-a499-4a5b-9f14-68f81d45c541 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-a5c727cd-a499-4a5b-9f14-68f81d45c541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:43.284999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:43.285915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:43.286744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:43.286744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:43.286744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:43.286744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 203/425] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:43.291751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06e48c-2fc4-4a1c-a094-569b8f7184bf None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:43.369014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:43.435168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06e48c-2fc4-4a1c-a094-569b8f7184bf None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:43.486464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65306be6-df76-4a3a-b8e3-689b9cc65eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:43.487084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65306be6-df76-4a3a-b8e3-689b9cc65eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:43.487365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65306be6-df76-4a3a-b8e3-689b9cc65eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:43.487634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65306be6-df76-4a3a-b8e3-689b9cc65eb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:43.488304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65306be6-df76-4a3a-b8e3-689b9cc65eb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:43.488600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65306be6-df76-4a3a-b8e3-689b9cc65eb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:43.489985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65306be6-df76-4a3a-b8e3-689b9cc65eb4 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:43.492408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c727cd-a499-4a5b-9f14-68f81d45c541 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:43.502798 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 223/426] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:43.508593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65306be6-df76-4a3a-b8e3-689b9cc65eb4 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:43.623911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c727cd-a499-4a5b-9f14-68f81d45c541 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:43.668136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:43.689050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65306be6-df76-4a3a-b8e3-689b9cc65eb4 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:43.701834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d11328-31a9-421a-bd6e-200a22d8e6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:43.702544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d11328-31a9-421a-bd6e-200a22d8e6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:43.702825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d11328-31a9-421a-bd6e-200a22d8e6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:43.703095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d11328-31a9-421a-bd6e-200a22d8e6d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:43.707140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:43.719461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d11328-31a9-421a-bd6e-200a22d8e6d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-f3d11328-31a9-421a-bd6e-200a22d8e6d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:43.726425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:43.727322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:43.728142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:43.730760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 204/427] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:43.732256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512fbeb-0dab-459c-8b58-27f03283d51a None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:43.732588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227495b-147b-4e54-a522-436da433ed3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:43.732980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512fbeb-0dab-459c-8b58-27f03283d51a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:43.733458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512fbeb-0dab-459c-8b58-27f03283d51a None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:43.734085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227495b-147b-4e54-a522-436da433ed3b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:43.734369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227495b-147b-4e54-a522-436da433ed3b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:43.734639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227495b-147b-4e54-a522-436da433ed3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:43.739949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 205/428] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:43.741356 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512fbeb-0dab-459c-8b58-27f03283d51a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:43.745216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227495b-147b-4e54-a522-436da433ed3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:43.746843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227495b-147b-4e54-a522-436da433ed3b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-7227495b-147b-4e54-a522-436da433ed3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:43.766438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:43.767348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:43.768267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 224/429] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:42:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:43.773967 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b0e14-7b04-4eb8-96be-80e16f596c49 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:43.774253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b0e14-7b04-4eb8-96be-80e16f596c49 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:43.774524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b0e14-7b04-4eb8-96be-80e16f596c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:43.795276 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-067b0e14-7b04-4eb8-96be-80e16f596c49 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:43.796509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-067b0e14-7b04-4eb8-96be-80e16f596c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:43.859830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512fbeb-0dab-459c-8b58-27f03283d51a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:43.889412 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 225/430] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:43.896027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b0e14-7b04-4eb8-96be-80e16f596c49 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:43.913008 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec8f56-692e-4809-b002-57f68cabba9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:43.914518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec8f56-692e-4809-b002-57f68cabba9e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:43.915025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec8f56-692e-4809-b002-57f68cabba9e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:43.915491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec8f56-692e-4809-b002-57f68cabba9e None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:43.916408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec8f56-692e-4809-b002-57f68cabba9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:43.916944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec8f56-692e-4809-b002-57f68cabba9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:43.918651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec8f56-692e-4809-b002-57f68cabba9e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:43.938927 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 206/431] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:43.945723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec8f56-692e-4809-b002-57f68cabba9e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:44.008690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:44.012935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b0e14-7b04-4eb8-96be-80e16f596c49 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:44.028593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:44.076352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeacc38-2128-4660-8eca-0a3f4a4551d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:44.077074 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeacc38-2128-4660-8eca-0a3f4a4551d6 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.077416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeacc38-2128-4660-8eca-0a3f4a4551d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.078038 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeacc38-2128-4660-8eca-0a3f4a4551d6 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.078663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeacc38-2128-4660-8eca-0a3f4a4551d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:44.079031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeacc38-2128-4660-8eca-0a3f4a4551d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:44.080575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeacc38-2128-4660-8eca-0a3f4a4551d6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:44.101236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec8f56-692e-4809-b002-57f68cabba9e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:44.119527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 226/432] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Oct 14 17:42:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:44.134711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeacc38-2128-4660-8eca-0a3f4a4551d6 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:44.146874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158b7b1-3771-4bc9-98d3-a6cd51a7d9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:44.147843 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158b7b1-3771-4bc9-98d3-a6cd51a7d9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.148320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158b7b1-3771-4bc9-98d3-a6cd51a7d9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.148750 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158b7b1-3771-4bc9-98d3-a6cd51a7d9c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.161792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158b7b1-3771-4bc9-98d3-a6cd51a7d9c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:44.163738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158b7b1-3771-4bc9-98d3-a6cd51a7d9c5 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-d158b7b1-3771-4bc9-98d3-a6cd51a7d9c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:44.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:44.181326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:44.182187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:44.182187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:44.182187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:44.184284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 207/433] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:44.190163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158b7b1-3771-4bc9-98d3-a6cd51a7d9c5 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:44.291795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeacc38-2128-4660-8eca-0a3f4a4551d6 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:44.327384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:44.336035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51a48b-966e-478c-b6d0-48115a47f18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:44.336447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158b7b1-3771-4bc9-98d3-a6cd51a7d9c5 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:44.337133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51a48b-966e-478c-b6d0-48115a47f18e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.337627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51a48b-966e-478c-b6d0-48115a47f18e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.338348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51a48b-966e-478c-b6d0-48115a47f18e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.353804 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51a48b-966e-478c-b6d0-48115a47f18e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:44.355855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51a48b-966e-478c-b6d0-48115a47f18e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-5f51a48b-966e-478c-b6d0-48115a47f18e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:44.373400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:44.374398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:44.375279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:44.375279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:44.375279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:44.375279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 227/434] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:44.376275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089ac0f-206d-499f-bd2b-c0f5d0bdcd07 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.376550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089ac0f-206d-499f-bd2b-c0f5d0bdcd07 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.376816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089ac0f-206d-499f-bd2b-c0f5d0bdcd07 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.378286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 228/435] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:44.378853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089ac0f-206d-499f-bd2b-c0f5d0bdcd07 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:44.384161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d5011-d98b-4bdf-ba40-794cb3f27d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:44.384161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d5011-d98b-4bdf-ba40-794cb3f27d19 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.384161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d5011-d98b-4bdf-ba40-794cb3f27d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.384161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d5011-d98b-4bdf-ba40-794cb3f27d19 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.384659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d5011-d98b-4bdf-ba40-794cb3f27d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:44.384659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d5011-d98b-4bdf-ba40-794cb3f27d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:44.384659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d5011-d98b-4bdf-ba40-794cb3f27d19 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:44.395725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:44.407417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 208/436] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:44.408330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d5011-d98b-4bdf-ba40-794cb3f27d19 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:44.490677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089ac0f-206d-499f-bd2b-c0f5d0bdcd07 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:44.512280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d5011-d98b-4bdf-ba40-794cb3f27d19 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:44.525995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d1b95-3edf-4f73-8487-a144d80887c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:44.526647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d1b95-3edf-4f73-8487-a144d80887c0 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.526922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d1b95-3edf-4f73-8487-a144d80887c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.527187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d1b95-3edf-4f73-8487-a144d80887c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.537803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d1b95-3edf-4f73-8487-a144d80887c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-943d1b95-3edf-4f73-8487-a144d80887c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:44.542657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:44.543571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:44.544451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 229/437] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:44.545121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01c272-111e-4133-a731-0f44196946fe None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.545121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01c272-111e-4133-a731-0f44196946fe None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.545335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01c272-111e-4133-a731-0f44196946fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.549651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38134d13-e3e1-455b-a153-868d8ce8dbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:44.550261 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38134d13-e3e1-455b-a153-868d8ce8dbce None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.550535 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38134d13-e3e1-455b-a153-868d8ce8dbce None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.550804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38134d13-e3e1-455b-a153-868d8ce8dbce None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.551639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38134d13-e3e1-455b-a153-868d8ce8dbce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:44.551977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38134d13-e3e1-455b-a153-868d8ce8dbce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:44.554347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38134d13-e3e1-455b-a153-868d8ce8dbce None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:44.562617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7c01c272-111e-4133-a731-0f44196946fe None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:44.563797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7c01c272-111e-4133-a731-0f44196946fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:44.566041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 209/438] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:44.567001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38134d13-e3e1-455b-a153-868d8ce8dbce None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:44.585870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:44.626005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:44.631253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 230/439] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:44.632953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18064-a98d-4d07-9d50-8ee72edbe5af None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.633234 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18064-a98d-4d07-9d50-8ee72edbe5af None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.633503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18064-a98d-4d07-9d50-8ee72edbe5af None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.639022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 231/440] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:44.639022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf18064-a98d-4d07-9d50-8ee72edbe5af None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:44.674073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38134d13-e3e1-455b-a153-868d8ce8dbce None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:44.711805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1702d19-914c-4b20-8668-07725abd8d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:44.712443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1702d19-914c-4b20-8668-07725abd8d70 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.712714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1702d19-914c-4b20-8668-07725abd8d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.712981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1702d19-914c-4b20-8668-07725abd8d70 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.720389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1702d19-914c-4b20-8668-07725abd8d70 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:44.721822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1702d19-914c-4b20-8668-07725abd8d70 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-e1702d19-914c-4b20-8668-07725abd8d70 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:44.733030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:44.734105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 210/441] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:44.735016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1702d19-914c-4b20-8668-07725abd8d70 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:44.752452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf18064-a98d-4d07-9d50-8ee72edbe5af None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:44.788682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff768e7-0a61-4da1-a962-18a10cc4f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:44.789286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff768e7-0a61-4da1-a962-18a10cc4f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.789554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff768e7-0a61-4da1-a962-18a10cc4f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.789819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff768e7-0a61-4da1-a962-18a10cc4f9f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.790482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff768e7-0a61-4da1-a962-18a10cc4f9f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:44.790774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff768e7-0a61-4da1-a962-18a10cc4f9f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:44.792168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff768e7-0a61-4da1-a962-18a10cc4f9f0 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:44.800345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 232/442] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:44.801435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa779f-c186-41fd-ae59-1fba573e6667 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.801720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa779f-c186-41fd-ae59-1fba573e6667 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.801990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa779f-c186-41fd-ae59-1fba573e6667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.820529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-6cfa779f-c186-41fd-ae59-1fba573e6667 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:44.821786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6cfa779f-c186-41fd-ae59-1fba573e6667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:44.822270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:44.849037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1702d19-914c-4b20-8668-07725abd8d70 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:44.879649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:44.888083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202f28f-6362-4b9e-a6fc-0c1af3104f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:44.889015 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202f28f-6362-4b9e-a6fc-0c1af3104f57 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:44.889293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202f28f-6362-4b9e-a6fc-0c1af3104f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:44.889569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202f28f-6362-4b9e-a6fc-0c1af3104f57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:44.892321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 233/443] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:44.896405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202f28f-6362-4b9e-a6fc-0c1af3104f57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:44.897059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa779f-c186-41fd-ae59-1fba573e6667 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:44.898025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202f28f-6362-4b9e-a6fc-0c1af3104f57 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-3202f28f-6362-4b9e-a6fc-0c1af3104f57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:44.909649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:44.910695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:44.911580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:44.911580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:44.911580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:44.911580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:44.911580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:44.911580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:44.911580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:44.911580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:44.911580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 211/444] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:44.912080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202f28f-6362-4b9e-a6fc-0c1af3104f57 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.015770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa779f-c186-41fd-ae59-1fba573e6667 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.018972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202f28f-6362-4b9e-a6fc-0c1af3104f57 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.054602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2776eb6-6a93-42e0-a7fb-68fd0a72b8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.055126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2776eb6-6a93-42e0-a7fb-68fd0a72b8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.055392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2776eb6-6a93-42e0-a7fb-68fd0a72b8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.055656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2776eb6-6a93-42e0-a7fb-68fd0a72b8f1 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.056403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2776eb6-6a93-42e0-a7fb-68fd0a72b8f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:45.056502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569032c-0cbe-4357-b797-2a63bf575e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.056692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2776eb6-6a93-42e0-a7fb-68fd0a72b8f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:45.057114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569032c-0cbe-4357-b797-2a63bf575e0f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.057384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569032c-0cbe-4357-b797-2a63bf575e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.057646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569032c-0cbe-4357-b797-2a63bf575e0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.058140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2776eb6-6a93-42e0-a7fb-68fd0a72b8f1 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:45.059529 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:45.069700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569032c-0cbe-4357-b797-2a63bf575e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:45.071135 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569032c-0cbe-4357-b797-2a63bf575e0f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:45.076458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 234/445] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Oct 14 17:42:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:45.077263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2776eb6-6a93-42e0-a7fb-68fd0a72b8f1 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-6569032c-0cbe-4357-b797-2a63bf575e0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:45.091469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:45.092514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:45.093382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:45.093382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:45.093382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:45.093382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 212/446] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:45.093382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569032c-0cbe-4357-b797-2a63bf575e0f None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.137259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:45.189290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2776eb6-6a93-42e0-a7fb-68fd0a72b8f1 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.199068 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569032c-0cbe-4357-b797-2a63bf575e0f None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.222514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd4dec-913d-4362-948c-2673df2e0942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.223120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd4dec-913d-4362-948c-2673df2e0942 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.223391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd4dec-913d-4362-948c-2673df2e0942 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.223652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd4dec-913d-4362-948c-2673df2e0942 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.232892 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f769be-fdd0-478b-bead-f3fa0fe3d788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.233332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd4dec-913d-4362-948c-2673df2e0942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:45.233495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f769be-fdd0-478b-bead-f3fa0fe3d788 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.233758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f769be-fdd0-478b-bead-f3fa0fe3d788 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.234026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f769be-fdd0-478b-bead-f3fa0fe3d788 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.234709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f769be-fdd0-478b-bead-f3fa0fe3d788 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:45.235010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f769be-fdd0-478b-bead-f3fa0fe3d788 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:45.236465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f769be-fdd0-478b-bead-f3fa0fe3d788 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-b5cd4dec-913d-4362-948c-2673df2e0942 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:45.238071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:45.239107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:45.240244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 235/447] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:45.240953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd4dec-913d-4362-948c-2673df2e0942 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.248887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 213/448] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:45.249578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f769be-fdd0-478b-bead-f3fa0fe3d788 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.296284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:45.341706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd4dec-913d-4362-948c-2673df2e0942 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.352570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f769be-fdd0-478b-bead-f3fa0fe3d788 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.352834 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:45.375989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72409141-4379-4c80-9890-72c40b8542c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.376582 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72409141-4379-4c80-9890-72c40b8542c0 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.376862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72409141-4379-4c80-9890-72c40b8542c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.377127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72409141-4379-4c80-9890-72c40b8542c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.387994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b10475-d431-4abb-8af1-22d74fe6853e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.388590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b10475-d431-4abb-8af1-22d74fe6853e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.388869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b10475-d431-4abb-8af1-22d74fe6853e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.389141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b10475-d431-4abb-8af1-22d74fe6853e None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.389775 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b10475-d431-4abb-8af1-22d74fe6853e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:45.390069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b10475-d431-4abb-8af1-22d74fe6853e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:45.391451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b10475-d431-4abb-8af1-22d74fe6853e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:45.403210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 214/449] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:45.404191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b10475-d431-4abb-8af1-22d74fe6853e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.445989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72409141-4379-4c80-9890-72c40b8542c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:45.447444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72409141-4379-4c80-9890-72c40b8542c0 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:45.502871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b10475-d431-4abb-8af1-22d74fe6853e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.510093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:45.539029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd1128-446a-4b19-86d4-bd355371acfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.539641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd1128-446a-4b19-86d4-bd355371acfa None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.540055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd1128-446a-4b19-86d4-bd355371acfa None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.540340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd1128-446a-4b19-86d4-bd355371acfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.543573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 236/450] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:45.544424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72409141-4379-4c80-9890-72c40b8542c0 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.550720 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd1128-446a-4b19-86d4-bd355371acfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-dafd1128-446a-4b19-86d4-bd355371acfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:45.555627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:45.556725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:45.557553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:45.558283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:45.558361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 215/451] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:45.558361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd1128-446a-4b19-86d4-bd355371acfa None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.647799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72409141-4379-4c80-9890-72c40b8542c0 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.660359 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd1128-446a-4b19-86d4-bd355371acfa None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.681222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e5fa0-b6f8-46da-909f-ba70f69fa6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.681903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e5fa0-b6f8-46da-909f-ba70f69fa6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.682180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e5fa0-b6f8-46da-909f-ba70f69fa6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.682443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e5fa0-b6f8-46da-909f-ba70f69fa6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.683101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e5fa0-b6f8-46da-909f-ba70f69fa6c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:45.683398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e5fa0-b6f8-46da-909f-ba70f69fa6c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:45.684787 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e5fa0-b6f8-46da-909f-ba70f69fa6c8 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:45.692751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 237/452] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:45.693312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e5fa0-b6f8-46da-909f-ba70f69fa6c8 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.693862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e6dd5-d1b5-4758-b23f-023f9b2f87c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.694525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6dd5-d1b5-4758-b23f-023f9b2f87c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.694799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6dd5-d1b5-4758-b23f-023f9b2f87c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.695070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6dd5-d1b5-4758-b23f-023f9b2f87c5 None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/9ec9c3fcf7964640a5eeabc01cd97617/roles/e1a3bea83ac349acb6d65b6afb5441d6` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.710553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e6dd5-d1b5-4758-b23f-023f9b2f87c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=9ec9c3fcf7964640a5eeabc01cd97617, role_id=e1a3bea83ac349acb6d65b6afb5441d6)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:45.712004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e6dd5-d1b5-4758-b23f-023f9b2f87c5 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:45.732229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 216/453] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:45 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/9ec9c3fcf7964640a5eeabc01cd97617/roles/e1a3bea83ac349acb6d65b6afb5441d6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:45.732229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e6dd5-d1b5-4758-b23f-023f9b2f87c5 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.749780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:45.772718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:45.808994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e5fa0-b6f8-46da-909f-ba70f69fa6c8 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.847270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e6dd5-d1b5-4758-b23f-023f9b2f87c5 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:45.847495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace3899-68cb-49ad-86bf-58b20f4ac1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.847974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace3899-68cb-49ad-86bf-58b20f4ac1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.848251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace3899-68cb-49ad-86bf-58b20f4ac1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.848522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace3899-68cb-49ad-86bf-58b20f4ac1c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.857683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace3899-68cb-49ad-86bf-58b20f4ac1c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:45.859170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace3899-68cb-49ad-86bf-58b20f4ac1c6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-7ace3899-68cb-49ad-86bf-58b20f4ac1c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:45.872337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:45.873321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:45.874224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 238/454] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Oct 14 17:42:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:45.874805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace3899-68cb-49ad-86bf-58b20f4ac1c6 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.888792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0622b-7e67-4e68-9bbc-19776275050d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:45.889502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0622b-7e67-4e68-9bbc-19776275050d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:45.889757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0622b-7e67-4e68-9bbc-19776275050d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:45.890031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0622b-7e67-4e68-9bbc-19776275050d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:45.903036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0622b-7e67-4e68-9bbc-19776275050d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:45.904638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0622b-7e67-4e68-9bbc-19776275050d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-4ee0622b-7e67-4e68-9bbc-19776275050d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:45.921481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:45.922628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:45.923484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:45.923484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:45.923484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:45.923484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 217/455] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:45.923484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0622b-7e67-4e68-9bbc-19776275050d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:45.989847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace3899-68cb-49ad-86bf-58b20f4ac1c6 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.003187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:46.028910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:46.035656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e5547-7d4c-401e-92b5-02f1dc33cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.036484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5547-7d4c-401e-92b5-02f1dc33cb6f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.036976 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5547-7d4c-401e-92b5-02f1dc33cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.037556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5547-7d4c-401e-92b5-02f1dc33cb6f None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.038287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e5547-7d4c-401e-92b5-02f1dc33cb6f None admin] RBAC: Adding query filter params (name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:46.038718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e5547-7d4c-401e-92b5-02f1dc33cb6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.040182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e5547-7d4c-401e-92b5-02f1dc33cb6f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.040461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0622b-7e67-4e68-9bbc-19776275050d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.053748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 239/456] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:46.053748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e5547-7d4c-401e-92b5-02f1dc33cb6f None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.080073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd576c4-893e-4ee7-b0b9-84e9d47cfea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.080704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd576c4-893e-4ee7-b0b9-84e9d47cfea3 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.080976 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd576c4-893e-4ee7-b0b9-84e9d47cfea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.081245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd576c4-893e-4ee7-b0b9-84e9d47cfea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.093126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd576c4-893e-4ee7-b0b9-84e9d47cfea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.094568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd576c4-893e-4ee7-b0b9-84e9d47cfea3 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-afd576c4-893e-4ee7-b0b9-84e9d47cfea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:46.108553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:46.109673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:46.110782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:46.110782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:46.110782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:46.114211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 218/457] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:46.115078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd576c4-893e-4ee7-b0b9-84e9d47cfea3 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.164524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e5547-7d4c-401e-92b5-02f1dc33cb6f None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.201142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c3e1c-064b-40a6-9638-dae7c392d6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.201751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3e1c-064b-40a6-9638-dae7c392d6bd None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.202027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3e1c-064b-40a6-9638-dae7c392d6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.202299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3e1c-064b-40a6-9638-dae7c392d6bd None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.203381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c3e1c-064b-40a6-9638-dae7c392d6bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:46.203677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c3e1c-064b-40a6-9638-dae7c392d6bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.205485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c3e1c-064b-40a6-9638-dae7c392d6bd None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.227944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 240/458] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:46.227944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c3e1c-064b-40a6-9638-dae7c392d6bd None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.233159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd576c4-893e-4ee7-b0b9-84e9d47cfea3 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.256050 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:46.274699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaf590-fca9-4acc-aded-d60ca2314e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.275393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf590-fca9-4acc-aded-d60ca2314e30 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.275668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf590-fca9-4acc-aded-d60ca2314e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.275964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf590-fca9-4acc-aded-d60ca2314e30 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.276664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdaf590-fca9-4acc-aded-d60ca2314e30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:46.276966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdaf590-fca9-4acc-aded-d60ca2314e30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.277679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:46.279418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdaf590-fca9-4acc-aded-d60ca2314e30 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.293642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 219/459] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:46.294398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaf590-fca9-4acc-aded-d60ca2314e30 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.332920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c3e1c-064b-40a6-9638-dae7c392d6bd None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.370120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa890931-c8f4-4b0f-9c70-2ae8b0a608fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.370880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa890931-c8f4-4b0f-9c70-2ae8b0a608fc None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.371255 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa890931-c8f4-4b0f-9c70-2ae8b0a608fc None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.371618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa890931-c8f4-4b0f-9c70-2ae8b0a608fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.382131 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa890931-c8f4-4b0f-9c70-2ae8b0a608fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.383695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa890931-c8f4-4b0f-9c70-2ae8b0a608fc None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-aa890931-c8f4-4b0f-9c70-2ae8b0a608fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:46.401558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:46.402676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:46.403641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:46.403641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:46.403641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:46.403641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 241/460] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:46.407921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa890931-c8f4-4b0f-9c70-2ae8b0a608fc None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.410941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaf590-fca9-4acc-aded-d60ca2314e30 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.447688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960c787-bf26-4a8d-a5ea-532b8d03bfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.448232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960c787-bf26-4a8d-a5ea-532b8d03bfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.448508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960c787-bf26-4a8d-a5ea-532b8d03bfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.448780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960c787-bf26-4a8d-a5ea-532b8d03bfc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.449532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960c787-bf26-4a8d-a5ea-532b8d03bfc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:46.449837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960c787-bf26-4a8d-a5ea-532b8d03bfc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.451205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960c787-bf26-4a8d-a5ea-532b8d03bfc6 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.463407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 220/461] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:46.464270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960c787-bf26-4a8d-a5ea-532b8d03bfc6 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.513354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:46.517702 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa890931-c8f4-4b0f-9c70-2ae8b0a608fc None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.521719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:46.560244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5def5d0-d153-4780-8ca3-b38024671165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.561258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5def5d0-d153-4780-8ca3-b38024671165 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.561541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5def5d0-d153-4780-8ca3-b38024671165 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.561811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5def5d0-d153-4780-8ca3-b38024671165 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.573915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960c787-bf26-4a8d-a5ea-532b8d03bfc6 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.611013 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aae86b-cc9b-4852-b010-9f62f0c310a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.611776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aae86b-cc9b-4852-b010-9f62f0c310a4 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.612085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aae86b-cc9b-4852-b010-9f62f0c310a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.612354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aae86b-cc9b-4852-b010-9f62f0c310a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.620981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aae86b-cc9b-4852-b010-9f62f0c310a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.622449 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aae86b-cc9b-4852-b010-9f62f0c310a4 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-41aae86b-cc9b-4852-b010-9f62f0c310a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:46.634524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:46.635407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:46.637065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:46.637065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:46.637065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:46.637065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:46.637065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:46.637065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:46.637065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:46.637065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:46.637065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 221/462] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:46.637065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aae86b-cc9b-4852-b010-9f62f0c310a4 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.642527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5def5d0-d153-4780-8ca3-b38024671165 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.644074 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5def5d0-d153-4780-8ca3-b38024671165 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.743600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:46.747023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aae86b-cc9b-4852-b010-9f62f0c310a4 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.752865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 242/463] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:46.752865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5def5d0-d153-4780-8ca3-b38024671165 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.754169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:46.788709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815debf4-f4f9-4470-aaa6-0f6b23b9d25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.789314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815debf4-f4f9-4470-aaa6-0f6b23b9d25b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.789584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815debf4-f4f9-4470-aaa6-0f6b23b9d25b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.789851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815debf4-f4f9-4470-aaa6-0f6b23b9d25b None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.790478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815debf4-f4f9-4470-aaa6-0f6b23b9d25b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:46.790830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815debf4-f4f9-4470-aaa6-0f6b23b9d25b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.792258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815debf4-f4f9-4470-aaa6-0f6b23b9d25b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.804439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 222/464] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:46.805195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815debf4-f4f9-4470-aaa6-0f6b23b9d25b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.864446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5def5d0-d153-4780-8ca3-b38024671165 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.903373 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fc4d3-3ab2-4934-8360-04e9a69e397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.904020 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fc4d3-3ab2-4934-8360-04e9a69e397f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.904297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fc4d3-3ab2-4934-8360-04e9a69e397f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.904561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fc4d3-3ab2-4934-8360-04e9a69e397f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.912337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fc4d3-3ab2-4934-8360-04e9a69e397f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.913446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815debf4-f4f9-4470-aaa6-0f6b23b9d25b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:46.913882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fc4d3-3ab2-4934-8360-04e9a69e397f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-d69fc4d3-3ab2-4934-8360-04e9a69e397f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:46.926239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:46.927209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 243/465] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:46.929756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fc4d3-3ab2-4934-8360-04e9a69e397f None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.955787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e6a10-e570-443b-8ee4-0506ee41b216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:46.956739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e6a10-e570-443b-8ee4-0506ee41b216 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:46.957162 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e6a10-e570-443b-8ee4-0506ee41b216 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:46.957601 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e6a10-e570-443b-8ee4-0506ee41b216 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:46.958554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e6a10-e570-443b-8ee4-0506ee41b216 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:46.959002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e6a10-e570-443b-8ee4-0506ee41b216 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:46.961370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e6a10-e570-443b-8ee4-0506ee41b216 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:46.973801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 223/466] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:46.974923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e6a10-e570-443b-8ee4-0506ee41b216 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:46.998707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:47.001874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:47.040949 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fc4d3-3ab2-4934-8360-04e9a69e397f None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:47.077703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e5e1b-2b23-4a5b-9314-6c42d1ed498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:47.078325 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e5e1b-2b23-4a5b-9314-6c42d1ed498f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:47.078600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e5e1b-2b23-4a5b-9314-6c42d1ed498f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:47.078866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e5e1b-2b23-4a5b-9314-6c42d1ed498f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:47.087323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e6a10-e570-443b-8ee4-0506ee41b216 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:47.089857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e5e1b-2b23-4a5b-9314-6c42d1ed498f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:47.091344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e5e1b-2b23-4a5b-9314-6c42d1ed498f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-a42e5e1b-2b23-4a5b-9314-6c42d1ed498f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:47.105889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:47.106819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:47.107679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:47.107679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:47.107679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:47.107679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 244/467] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:47.108536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e5e1b-2b23-4a5b-9314-6c42d1ed498f None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:47.126785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7bbbb-f413-4c4a-a3b5-18336737dcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:47.127400 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7bbbb-f413-4c4a-a3b5-18336737dcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:47.127678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7bbbb-f413-4c4a-a3b5-18336737dcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:47.127989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7bbbb-f413-4c4a-a3b5-18336737dcb9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:47.136698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7bbbb-f413-4c4a-a3b5-18336737dcb9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:47.138391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7bbbb-f413-4c4a-a3b5-18336737dcb9 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-99b7bbbb-f413-4c4a-a3b5-18336737dcb9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:47.150472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:47.151546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:47.152694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:47.152694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:47.152694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:47.152694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:47.152694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:47.152694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:47.152694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:47.152694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:47.152694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 17:42:47.152694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:47.153881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 224/468] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:47.156701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7bbbb-f413-4c4a-a3b5-18336737dcb9 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:47.220061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e5e1b-2b23-4a5b-9314-6c42d1ed498f None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:47.235666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:47.242676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:47.254509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23578ef0-8757-4b6a-aa92-947bed47b7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:47.255052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23578ef0-8757-4b6a-aa92-947bed47b7bb None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:47.255382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23578ef0-8757-4b6a-aa92-947bed47b7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:47.255651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23578ef0-8757-4b6a-aa92-947bed47b7bb None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:47.256435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23578ef0-8757-4b6a-aa92-947bed47b7bb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:47.257314 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23578ef0-8757-4b6a-aa92-947bed47b7bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:47.258927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23578ef0-8757-4b6a-aa92-947bed47b7bb None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:47.266376 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7bbbb-f413-4c4a-a3b5-18336737dcb9 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:47.276817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 245/469] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Oct 14 17:42:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:47.277714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23578ef0-8757-4b6a-aa92-947bed47b7bb None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:47.305898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbb21a-e897-4011-a91c-b84c97eea5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:47.306669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbb21a-e897-4011-a91c-b84c97eea5fd None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:47.306784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbb21a-e897-4011-a91c-b84c97eea5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:47.307013 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbb21a-e897-4011-a91c-b84c97eea5fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:47.307739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbb21a-e897-4011-a91c-b84c97eea5fd None admin] RBAC: Adding query filter params (role.id=e1a3bea83ac349acb6d65b6afb5441d6, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=ae3566d85c9c4c559ba672269ab38f09) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:47.308077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbb21a-e897-4011-a91c-b84c97eea5fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:47.309473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbb21a-e897-4011-a91c-b84c97eea5fd None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:47.319946 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 225/470] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:42:47 2022] GET /identity/v3/role_assignments?user.id=ae3566d85c9c4c559ba672269ab38f09&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=e1a3bea83ac349acb6d65b6afb5441d6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:47.319946 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbb21a-e897-4011-a91c-b84c97eea5fd None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:47.419077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23578ef0-8757-4b6a-aa92-947bed47b7bb None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:47.464792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbb21a-e897-4011-a91c-b84c97eea5fd None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:47.470334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee358646-7158-4a90-ad35-e704e810c887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:47.471107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee358646-7158-4a90-ad35-e704e810c887 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:47.471482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee358646-7158-4a90-ad35-e704e810c887 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:47.471849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee358646-7158-4a90-ad35-e704e810c887 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:47.472644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee358646-7158-4a90-ad35-e704e810c887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:47.473706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee358646-7158-4a90-ad35-e704e810c887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:47.476473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee358646-7158-4a90-ad35-e704e810c887 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:47.492405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 246/471] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:47.493348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee358646-7158-4a90-ad35-e704e810c887 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:47.518737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe16250-53f8-4872-9e21-a9bc0cb613e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:47.519418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe16250-53f8-4872-9e21-a9bc0cb613e9 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:47.519859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe16250-53f8-4872-9e21-a9bc0cb613e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:47.520258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe16250-53f8-4872-9e21-a9bc0cb613e9 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:47.521086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe16250-53f8-4872-9e21-a9bc0cb613e9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:47.521487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe16250-53f8-4872-9e21-a9bc0cb613e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:47.523796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe16250-53f8-4872-9e21-a9bc0cb613e9 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:47.533605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:47.567410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 226/472] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Oct 14 17:42:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:47.568585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe16250-53f8-4872-9e21-a9bc0cb613e9 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:47.572327 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:47.615514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee358646-7158-4a90-ad35-e704e810c887 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:47.660797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c37a-80b9-4536-b188-dc023bf126f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:47.662242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c37a-80b9-4536-b188-dc023bf126f5 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:47.662516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c37a-80b9-4536-b188-dc023bf126f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:47.663319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c37a-80b9-4536-b188-dc023bf126f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:47.700412 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe16250-53f8-4872-9e21-a9bc0cb613e9 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:47.743686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3f166-d107-4d97-bcac-2929222a21e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:47.744586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f166-d107-4d97-bcac-2929222a21e3 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:47.745274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f166-d107-4d97-bcac-2929222a21e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:47.745558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f166-d107-4d97-bcac-2929222a21e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:47.754568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3f166-d107-4d97-bcac-2929222a21e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:47.756011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c37a-80b9-4536-b188-dc023bf126f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:47.756353 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3f166-d107-4d97-bcac-2929222a21e3 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:47.758334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c37a-80b9-4536-b188-dc023bf126f5 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-25d3f166-d107-4d97-bcac-2929222a21e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:47.770287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:47.771159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:47.772220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:47.772220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:47.772220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:47.772220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:47.772220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:47.772220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:47.772220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:47.772220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:47.772220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 227/473] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:47.776701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3f166-d107-4d97-bcac-2929222a21e3 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:47.808221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:47.860278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:47.884049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 247/474] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:47.884969 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c37a-80b9-4536-b188-dc023bf126f5 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:47.912343 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3f166-d107-4d97-bcac-2929222a21e3 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:47.950568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797daaf2-ce2c-4fd6-b0fa-53a5bd29493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:47.951232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797daaf2-ce2c-4fd6-b0fa-53a5bd29493e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:47.951461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797daaf2-ce2c-4fd6-b0fa-53a5bd29493e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:47.951696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797daaf2-ce2c-4fd6-b0fa-53a5bd29493e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:47.989290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c37a-80b9-4536-b188-dc023bf126f5 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:48.025388 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797daaf2-ce2c-4fd6-b0fa-53a5bd29493e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:48.027057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797daaf2-ce2c-4fd6-b0fa-53a5bd29493e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:48.027167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e55fe-11d2-4e6d-8d48-5a5151de4ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:48.027668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e55fe-11d2-4e6d-8d48-5a5151de4ade None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:48.027976 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e55fe-11d2-4e6d-8d48-5a5151de4ade None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:48.028242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e55fe-11d2-4e6d-8d48-5a5151de4ade None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:48.028925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e55fe-11d2-4e6d-8d48-5a5151de4ade None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:48.029234 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e55fe-11d2-4e6d-8d48-5a5151de4ade None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:48.030694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e55fe-11d2-4e6d-8d48-5a5151de4ade None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:48.032523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:48.043532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 248/475] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:48.044508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e55fe-11d2-4e6d-8d48-5a5151de4ade None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:48.098096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:48.143998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 228/476] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:48.145421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3a645-f99f-471c-afc7-9b37252678f5 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:48.145773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3a645-f99f-471c-afc7-9b37252678f5 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:48.146113 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3a645-f99f-471c-afc7-9b37252678f5 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:48.152096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 229/477] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:48.152096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3a645-f99f-471c-afc7-9b37252678f5 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:48.158629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e55fe-11d2-4e6d-8d48-5a5151de4ade None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:48.195327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a56c3-4fa1-4eeb-8473-d6065dec3d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:48.195993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a56c3-4fa1-4eeb-8473-d6065dec3d30 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:48.196269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a56c3-4fa1-4eeb-8473-d6065dec3d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:48.196540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a56c3-4fa1-4eeb-8473-d6065dec3d30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:48.207440 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a56c3-4fa1-4eeb-8473-d6065dec3d30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:48.208906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a56c3-4fa1-4eeb-8473-d6065dec3d30 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-e08a56c3-4fa1-4eeb-8473-d6065dec3d30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:48.224225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:48.225254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:48.226115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:48.226115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:48.226115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:48.226115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 249/478] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:48.226115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a56c3-4fa1-4eeb-8473-d6065dec3d30 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:48.266564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3a645-f99f-471c-afc7-9b37252678f5 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:48.275263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:48.302262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a508b8d-6ecd-4351-9a89-d7e55ea3a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:48.302866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a508b8d-6ecd-4351-9a89-d7e55ea3a3bf None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:48.303139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a508b8d-6ecd-4351-9a89-d7e55ea3a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:48.303458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a508b8d-6ecd-4351-9a89-d7e55ea3a3bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:48.304211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a508b8d-6ecd-4351-9a89-d7e55ea3a3bf None admin] RBAC: Adding query filter params (role.id=e1a3bea83ac349acb6d65b6afb5441d6, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=e42968016de8470c933655e921fb3394) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:48.304515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a508b8d-6ecd-4351-9a89-d7e55ea3a3bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:48.305942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a508b8d-6ecd-4351-9a89-d7e55ea3a3bf None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:48.312208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 230/479] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:42:48 2022] GET /identity/v3/role_assignments?user.id=e42968016de8470c933655e921fb3394&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=e1a3bea83ac349acb6d65b6afb5441d6 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:48.313418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f04ea-5443-44bb-9df3-f78b649cf676 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:48.313728 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f04ea-5443-44bb-9df3-f78b649cf676 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:48.314052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f04ea-5443-44bb-9df3-f78b649cf676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:48.335737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-5d4f04ea-5443-44bb-9df3-f78b649cf676 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:48.337187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5d4f04ea-5443-44bb-9df3-f78b649cf676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:48.354539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a56c3-4fa1-4eeb-8473-d6065dec3d30 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:48.366684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:48.412595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a886f8-1118-40f6-98dc-8f0ed86f24be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:48.413644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a886f8-1118-40f6-98dc-8f0ed86f24be None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:48.414068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a886f8-1118-40f6-98dc-8f0ed86f24be None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:48.415169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a886f8-1118-40f6-98dc-8f0ed86f24be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:48.431175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a886f8-1118-40f6-98dc-8f0ed86f24be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:48.439919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 231/480] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:48.439919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f04ea-5443-44bb-9df3-f78b649cf676 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:48.441350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a886f8-1118-40f6-98dc-8f0ed86f24be None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-37a886f8-1118-40f6-98dc-8f0ed86f24be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:48.460819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:48.461985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:48.463178 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:48.463178 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:48.463178 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:48.464573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 250/481] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:48.471110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a886f8-1118-40f6-98dc-8f0ed86f24be None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:48.586821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f04ea-5443-44bb-9df3-f78b649cf676 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:48.603253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:48.608420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a886f8-1118-40f6-98dc-8f0ed86f24be None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:48.627203 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847954a-7d11-4f83-9bc0-a5f28ebd6128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:48.628002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847954a-7d11-4f83-9bc0-a5f28ebd6128 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:48.628377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847954a-7d11-4f83-9bc0-a5f28ebd6128 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:48.628744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847954a-7d11-4f83-9bc0-a5f28ebd6128 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:48.629581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847954a-7d11-4f83-9bc0-a5f28ebd6128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:48.629972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847954a-7d11-4f83-9bc0-a5f28ebd6128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:48.632394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847954a-7d11-4f83-9bc0-a5f28ebd6128 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:48.645703 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 232/482] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:48.650671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c3dd7-b43e-4e2b-a260-66c6cf1c4167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:48.651280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c3dd7-b43e-4e2b-a260-66c6cf1c4167 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:48.651554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c3dd7-b43e-4e2b-a260-66c6cf1c4167 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:48.651664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847954a-7d11-4f83-9bc0-a5f28ebd6128 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:48.651819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c3dd7-b43e-4e2b-a260-66c6cf1c4167 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:48.652495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c3dd7-b43e-4e2b-a260-66c6cf1c4167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:48.652792 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c3dd7-b43e-4e2b-a260-66c6cf1c4167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:48.654182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c3dd7-b43e-4e2b-a260-66c6cf1c4167 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:48.673103 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:48.673608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 251/483] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:48.679154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c3dd7-b43e-4e2b-a260-66c6cf1c4167 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:48.794494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847954a-7d11-4f83-9bc0-a5f28ebd6128 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:48.799394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c3dd7-b43e-4e2b-a260-66c6cf1c4167 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:48.837415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b2dd0-34e3-4f83-9813-a10f2401d38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:48.838374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b2dd0-34e3-4f83-9813-a10f2401d38b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:48.838794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b2dd0-34e3-4f83-9813-a10f2401d38b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:48.839222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b2dd0-34e3-4f83-9813-a10f2401d38b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:48.842853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274ba86-db82-466c-b5c9-610f6f83a9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:48.843480 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274ba86-db82-466c-b5c9-610f6f83a9ce None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:48.843756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274ba86-db82-466c-b5c9-610f6f83a9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:48.844052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274ba86-db82-466c-b5c9-610f6f83a9ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:48.847413 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b2dd0-34e3-4f83-9813-a10f2401d38b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:48.849046 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b2dd0-34e3-4f83-9813-a10f2401d38b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:48.852341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274ba86-db82-466c-b5c9-610f6f83a9ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:48.854143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274ba86-db82-466c-b5c9-610f6f83a9ce None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-8f0b2dd0-34e3-4f83-9813-a10f2401d38b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:48.862726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:48.863770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:48.864818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:48.864818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:48.864818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:48.864818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:48.864818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:48.864818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:48.864818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:48.864818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-2274ba86-db82-466c-b5c9-610f6f83a9ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:48.865972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:48.866749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:48.867747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:48.867747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:48.867747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:48.867747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:48.867747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:48.867747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:48.867747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:48.867747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:48.867747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 252/484] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:48.868706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274ba86-db82-466c-b5c9-610f6f83a9ce None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:48.869216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 233/485] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:48.873906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b2dd0-34e3-4f83-9813-a10f2401d38b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:48.892236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:48.963590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:48.989161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b2dd0-34e3-4f83-9813-a10f2401d38b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:48.989721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274ba86-db82-466c-b5c9-610f6f83a9ce None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:49.026470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dbb52-775a-4924-816f-fb8903864a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:49.027233 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dbb52-775a-4924-816f-fb8903864a69 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:49.027568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dbb52-775a-4924-816f-fb8903864a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:49.027850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dbb52-775a-4924-816f-fb8903864a69 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:49.029029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8dbb52-775a-4924-816f-fb8903864a69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:49.029365 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8dbb52-775a-4924-816f-fb8903864a69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:49.029812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd53a1c-2e11-48ad-af85-20201cffb231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:49.030706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd53a1c-2e11-48ad-af85-20201cffb231 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:49.031124 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8dbb52-775a-4924-816f-fb8903864a69 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:49.031389 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd53a1c-2e11-48ad-af85-20201cffb231 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:49.031821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd53a1c-2e11-48ad-af85-20201cffb231 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:49.043831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd53a1c-2e11-48ad-af85-20201cffb231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:49.044838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 234/486] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:49.045742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dbb52-775a-4924-816f-fb8903864a69 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-bbd53a1c-2e11-48ad-af85-20201cffb231 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:49.050563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:49.051458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:49.052279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:49.054575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 253/487] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:49.055560 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd53a1c-2e11-48ad-af85-20201cffb231 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:49.148361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:49.156415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dbb52-775a-4924-816f-fb8903864a69 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:49.174689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd53a1c-2e11-48ad-af85-20201cffb231 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:49.191470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b975cb0-5aeb-467a-aae9-453b37d1eef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:49.192160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b975cb0-5aeb-467a-aae9-453b37d1eef5 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:49.192442 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b975cb0-5aeb-467a-aae9-453b37d1eef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:49.192803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b975cb0-5aeb-467a-aae9-453b37d1eef5 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:49.193439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b975cb0-5aeb-467a-aae9-453b37d1eef5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:49.193749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b975cb0-5aeb-467a-aae9-453b37d1eef5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:49.196086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b975cb0-5aeb-467a-aae9-453b37d1eef5 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:49.198778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:49.210669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 235/488] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:49.211564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b975cb0-5aeb-467a-aae9-453b37d1eef5 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:49.223164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a90f40-dd60-4fa2-8169-407259a86277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:49.223776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a90f40-dd60-4fa2-8169-407259a86277 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:49.224129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a90f40-dd60-4fa2-8169-407259a86277 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:49.224461 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a90f40-dd60-4fa2-8169-407259a86277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:49.225688 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a90f40-dd60-4fa2-8169-407259a86277 None admin] RBAC: Adding query filter params (role.id=e1a3bea83ac349acb6d65b6afb5441d6, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=3c17f274e4c14f2f99650844642437a2) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:49.226010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a90f40-dd60-4fa2-8169-407259a86277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:49.227851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a90f40-dd60-4fa2-8169-407259a86277 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:49.236019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 254/489] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:42:49 2022] GET /identity/v3/role_assignments?user.id=3c17f274e4c14f2f99650844642437a2&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=e1a3bea83ac349acb6d65b6afb5441d6 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:49.237248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a90f40-dd60-4fa2-8169-407259a86277 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:49.371603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b975cb0-5aeb-467a-aae9-453b37d1eef5 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:49.423605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a90f40-dd60-4fa2-8169-407259a86277 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:49.429939 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1f4b0-9460-4c9b-82ae-72c61e5ea4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:49.430827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1f4b0-9460-4c9b-82ae-72c61e5ea4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:49.431245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1f4b0-9460-4c9b-82ae-72c61e5ea4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:49.431644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1f4b0-9460-4c9b-82ae-72c61e5ea4e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:49.432495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1f4b0-9460-4c9b-82ae-72c61e5ea4e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:49.432952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1f4b0-9460-4c9b-82ae-72c61e5ea4e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:49.434595 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1f4b0-9460-4c9b-82ae-72c61e5ea4e7 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:49.443833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 236/490] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:49.449812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1f4b0-9460-4c9b-82ae-72c61e5ea4e7 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:49.464748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bde79-8d53-4707-a401-de36c013bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:49.465595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bde79-8d53-4707-a401-de36c013bb48 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:49.465900 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bde79-8d53-4707-a401-de36c013bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:49.466248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bde79-8d53-4707-a401-de36c013bb48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:49.467076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bde79-8d53-4707-a401-de36c013bb48 None admin] RBAC: Adding query filter params (role.id=e1a3bea83ac349acb6d65b6afb5441d6, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=8f51ee9e45fb4dda8d5b9740aa0ac828) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:49.467480 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bde79-8d53-4707-a401-de36c013bb48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:49.469202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bde79-8d53-4707-a401-de36c013bb48 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:49.476340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 255/491] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:42:49 2022] GET /identity/v3/role_assignments?user.id=8f51ee9e45fb4dda8d5b9740aa0ac828&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=e1a3bea83ac349acb6d65b6afb5441d6 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:49.524208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:49.574800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1f4b0-9460-4c9b-82ae-72c61e5ea4e7 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:49.614862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93122993-0935-43de-b22e-31c561a857ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:49.615497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93122993-0935-43de-b22e-31c561a857ba None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:49.615771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93122993-0935-43de-b22e-31c561a857ba None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:49.616321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93122993-0935-43de-b22e-31c561a857ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:49.628598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93122993-0935-43de-b22e-31c561a857ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:49.630141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93122993-0935-43de-b22e-31c561a857ba None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-93122993-0935-43de-b22e-31c561a857ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:49.654443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:49.655422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:49.657418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:49.657418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:49.657418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:49.657418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 237/492] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:49.662171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bde79-8d53-4707-a401-de36c013bb48 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:49.677690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:49.784951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2053c08-0675-46b4-aec6-cb05b36fd7c3 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:49.784951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2053c08-0675-46b4-aec6-cb05b36fd7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:49.784951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2053c08-0675-46b4-aec6-cb05b36fd7c3 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:49.784951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 238/493] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:49.790079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bd647-bb0a-4b1a-a67b-4b8cdb92cefb None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:49.790465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bd647-bb0a-4b1a-a67b-4b8cdb92cefb None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:49.790832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bd647-bb0a-4b1a-a67b-4b8cdb92cefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:49.795064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bde79-8d53-4707-a401-de36c013bb48 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:49.814045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-144bd647-bb0a-4b1a-a67b-4b8cdb92cefb None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:49.815408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-144bd647-bb0a-4b1a-a67b-4b8cdb92cefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:49.832764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f879479-2a04-44d5-92b4-93f349426ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:49.833764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f879479-2a04-44d5-92b4-93f349426ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:49.834044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f879479-2a04-44d5-92b4-93f349426ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:49.834616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f879479-2a04-44d5-92b4-93f349426ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:49.835273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f879479-2a04-44d5-92b4-93f349426ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:49.835570 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f879479-2a04-44d5-92b4-93f349426ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:49.837487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f879479-2a04-44d5-92b4-93f349426ca3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:49.854190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 256/494] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:49.860997 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f879479-2a04-44d5-92b4-93f349426ca3 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:49.899536 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 239/495] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:49.936949 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:49.974161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f879479-2a04-44d5-92b4-93f349426ca3 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:50.011171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3698f-8c68-4fe0-9c8d-506ec4c9c535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:50.011807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3698f-8c68-4fe0-9c8d-506ec4c9c535 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:50.012103 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3698f-8c68-4fe0-9c8d-506ec4c9c535 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:50.012388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3698f-8c68-4fe0-9c8d-506ec4c9c535 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:50.019978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3698f-8c68-4fe0-9c8d-506ec4c9c535 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:50.021422 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3698f-8c68-4fe0-9c8d-506ec4c9c535 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-1ba3698f-8c68-4fe0-9c8d-506ec4c9c535 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:50.033523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:50.034488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:50.036612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 257/496] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:42:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:50.041339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bd647-bb0a-4b1a-a67b-4b8cdb92cefb None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:50.085387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:50.147096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bd647-bb0a-4b1a-a67b-4b8cdb92cefb None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:50.182259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d02158-106c-47a8-be7d-7579e2617bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:50.182873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d02158-106c-47a8-be7d-7579e2617bdd None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:50.183147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d02158-106c-47a8-be7d-7579e2617bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:50.183419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d02158-106c-47a8-be7d-7579e2617bdd None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:50.184171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d02158-106c-47a8-be7d-7579e2617bdd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:50.184475 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d02158-106c-47a8-be7d-7579e2617bdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:50.185974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d02158-106c-47a8-be7d-7579e2617bdd None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:50.204324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 240/497] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Oct 14 17:42:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:50.210450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3698f-8c68-4fe0-9c8d-506ec4c9c535 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:50.235092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d02158-106c-47a8-be7d-7579e2617bdd None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:50.347004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d02158-106c-47a8-be7d-7579e2617bdd None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:50.388251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:50.388788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb3724-0130-475d-91d2-d41cc9342d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:50.389586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb3724-0130-475d-91d2-d41cc9342d6e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:50.390046 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb3724-0130-475d-91d2-d41cc9342d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:50.390608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb3724-0130-475d-91d2-d41cc9342d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:50.402207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb3724-0130-475d-91d2-d41cc9342d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:50.403714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb3724-0130-475d-91d2-d41cc9342d6e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-3cdb3724-0130-475d-91d2-d41cc9342d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:50.422689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:50.423685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:50.424597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:50.424597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:50.424597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:50.425182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 241/498] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:50.431169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb3724-0130-475d-91d2-d41cc9342d6e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:50.433188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3698f-8c68-4fe0-9c8d-506ec4c9c535 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:50.480906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:50.497125 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd4590-38c5-4694-9745-13be32c55610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:50.497771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd4590-38c5-4694-9745-13be32c55610 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:50.498010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd4590-38c5-4694-9745-13be32c55610 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:50.498280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd4590-38c5-4694-9745-13be32c55610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:50.591481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd4590-38c5-4694-9745-13be32c55610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:50.593142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd4590-38c5-4694-9745-13be32c55610 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:50.647191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb3724-0130-475d-91d2-d41cc9342d6e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:50.725404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 258/499] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 17:42:50.733000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd4590-38c5-4694-9745-13be32c55610 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:50.735536 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ba456-93ec-43fc-b798-6350d655c527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:50.736378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ba456-93ec-43fc-b798-6350d655c527 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:50.736867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ba456-93ec-43fc-b798-6350d655c527 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:50.737337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ba456-93ec-43fc-b798-6350d655c527 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:50.738176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ba456-93ec-43fc-b798-6350d655c527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:50.738671 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ba456-93ec-43fc-b798-6350d655c527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:50.740270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ba456-93ec-43fc-b798-6350d655c527 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:50.753248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 242/500] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:50.760653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ba456-93ec-43fc-b798-6350d655c527 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:50.783897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:50.784091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:50.872711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd4590-38c5-4694-9745-13be32c55610 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:50.935260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb98dea-5843-45d1-8da0-cdb9746cf820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:50.936267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb98dea-5843-45d1-8da0-cdb9746cf820 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:50.936691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb98dea-5843-45d1-8da0-cdb9746cf820 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:50.937131 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb98dea-5843-45d1-8da0-cdb9746cf820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:50.951199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb98dea-5843-45d1-8da0-cdb9746cf820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:50.953000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb98dea-5843-45d1-8da0-cdb9746cf820 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-1cb98dea-5843-45d1-8da0-cdb9746cf820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:50.985948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:50.987143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:50.988783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:50.988783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:50.988783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:50.995254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ba456-93ec-43fc-b798-6350d655c527 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:50.998297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 259/501] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:51.019976 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb98dea-5843-45d1-8da0-cdb9746cf820 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:51.066268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e1ee1-28d5-44ff-a445-92bd6662e8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:51.068423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e1ee1-28d5-44ff-a445-92bd6662e8a2 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:51.070139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e1ee1-28d5-44ff-a445-92bd6662e8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:51.070564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e1ee1-28d5-44ff-a445-92bd6662e8a2 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:51.071355 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e1ee1-28d5-44ff-a445-92bd6662e8a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:51.073634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e1ee1-28d5-44ff-a445-92bd6662e8a2 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:51.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 243/502] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:42:50 2022] POST /identity/v3/users => generated 254 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:42:51.115188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf29ca-8909-4024-a8e2-1e618dd755e0 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:51.117780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf29ca-8909-4024-a8e2-1e618dd755e0 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:51.118265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf29ca-8909-4024-a8e2-1e618dd755e0 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:51.121151 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 244/503] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:51.128557 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677f449-e414-4f70-b123-44c4c53ab473 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:51.129073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677f449-e414-4f70-b123-44c4c53ab473 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:51.129515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677f449-e414-4f70-b123-44c4c53ab473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:51.167266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-3677f449-e414-4f70-b123-44c4c53ab473 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:51.170274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:51.170516 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3677f449-e414-4f70-b123-44c4c53ab473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:51.236641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb98dea-5843-45d1-8da0-cdb9746cf820 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:51.243511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:51.253423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 245/504] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:51.320620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1234114-a15d-43ff-92e3-182ff3862a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:51.321237 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1234114-a15d-43ff-92e3-182ff3862a86 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:51.321514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1234114-a15d-43ff-92e3-182ff3862a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:51.321781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1234114-a15d-43ff-92e3-182ff3862a86 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:51.322435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1234114-a15d-43ff-92e3-182ff3862a86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:51.322735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1234114-a15d-43ff-92e3-182ff3862a86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:51.324134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1234114-a15d-43ff-92e3-182ff3862a86 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:51.337156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 260/505] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:51.342345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677f449-e414-4f70-b123-44c4c53ab473 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:51.452683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677f449-e414-4f70-b123-44c4c53ab473 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:51.489073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a79f-aa90-4dfc-9e3c-9441d2a3cf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:51.489626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a79f-aa90-4dfc-9e3c-9441d2a3cf32 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:51.489901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a79f-aa90-4dfc-9e3c-9441d2a3cf32 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:51.490171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a79f-aa90-4dfc-9e3c-9441d2a3cf32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:51.497737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09a79f-aa90-4dfc-9e3c-9441d2a3cf32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:51.499290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09a79f-aa90-4dfc-9e3c-9441d2a3cf32 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-cb09a79f-aa90-4dfc-9e3c-9441d2a3cf32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:51.511226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:51.512145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:51.513301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:51.513301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:51.513301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:51.513301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:51.513301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:51.513301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:51.513301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:51.513301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:51.513301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 246/506] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:51.520817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1234114-a15d-43ff-92e3-182ff3862a86 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:51.593111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a79f-aa90-4dfc-9e3c-9441d2a3cf32 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:51.659117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1234114-a15d-43ff-92e3-182ff3862a86 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:51.691473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:51.712945 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694ca2f-0e87-490f-b229-820fb3d4f42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:51.713906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694ca2f-0e87-490f-b229-820fb3d4f42a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:51.714411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694ca2f-0e87-490f-b229-820fb3d4f42a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:51.714879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694ca2f-0e87-490f-b229-820fb3d4f42a None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:51.715786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694ca2f-0e87-490f-b229-820fb3d4f42a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:51.716312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694ca2f-0e87-490f-b229-820fb3d4f42a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:51.718943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694ca2f-0e87-490f-b229-820fb3d4f42a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:51.732500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 261/507] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:51.734078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:51.739388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694ca2f-0e87-490f-b229-820fb3d4f42a None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:51.794687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a79f-aa90-4dfc-9e3c-9441d2a3cf32 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:51.884810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694ca2f-0e87-490f-b229-820fb3d4f42a None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:51.888746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96371e05-9d87-42d8-aa68-aa34296de849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:51.889401 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96371e05-9d87-42d8-aa68-aa34296de849 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:51.889680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96371e05-9d87-42d8-aa68-aa34296de849 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:51.889952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96371e05-9d87-42d8-aa68-aa34296de849 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:51.900917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96371e05-9d87-42d8-aa68-aa34296de849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-96371e05-9d87-42d8-aa68-aa34296de849 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:51.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:51.908157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:51.909025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 247/508] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:51.910208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60958-ed34-40fe-9b97-5a8d32fc3f08 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:51.910208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60958-ed34-40fe-9b97-5a8d32fc3f08 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:51.910373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60958-ed34-40fe-9b97-5a8d32fc3f08 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:51.914138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 248/509] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:51.917265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d60958-ed34-40fe-9b97-5a8d32fc3f08 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:51.922662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c0fb1-21cb-40ec-85ab-e96c2655d2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:51.923265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c0fb1-21cb-40ec-85ab-e96c2655d2cc None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:51.923535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c0fb1-21cb-40ec-85ab-e96c2655d2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:51.923799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c0fb1-21cb-40ec-85ab-e96c2655d2cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:51.924648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c0fb1-21cb-40ec-85ab-e96c2655d2cc None admin] RBAC: Adding query filter params (role.id=e1a3bea83ac349acb6d65b6afb5441d6, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=9ec9c3fcf7964640a5eeabc01cd97617) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:51.924954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c0fb1-21cb-40ec-85ab-e96c2655d2cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:51.926388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c0fb1-21cb-40ec-85ab-e96c2655d2cc None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:51.933894 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 262/510] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:42:51 2022] GET /identity/v3/role_assignments?user.id=9ec9c3fcf7964640a5eeabc01cd97617&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=e1a3bea83ac349acb6d65b6afb5441d6 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:51.935096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced4179-ad14-4929-b470-9627da9700e5 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:51.935440 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced4179-ad14-4929-b470-9627da9700e5 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:51.935710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced4179-ad14-4929-b470-9627da9700e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:51.957319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2ced4179-ad14-4929-b470-9627da9700e5 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:51.958532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2ced4179-ad14-4929-b470-9627da9700e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:52.028736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:52.035265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 263/511] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:52.109441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d60958-ed34-40fe-9b97-5a8d32fc3f08 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:52.140257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:52.163950 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5eec7-7f5d-4e9b-a4be-a1612022c1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:52.164880 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5eec7-7f5d-4e9b-a4be-a1612022c1ee None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:52.165230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5eec7-7f5d-4e9b-a4be-a1612022c1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:52.165653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5eec7-7f5d-4e9b-a4be-a1612022c1ee None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:52.166397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5eec7-7f5d-4e9b-a4be-a1612022c1ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:52.167991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5eec7-7f5d-4e9b-a4be-a1612022c1ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:52.169403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5eec7-7f5d-4e9b-a4be-a1612022c1ee None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:52.184020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 249/512] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:52.185540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced4179-ad14-4929-b470-9627da9700e5 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:52.326387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced4179-ad14-4929-b470-9627da9700e5 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:52.353392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5eec7-7f5d-4e9b-a4be-a1612022c1ee None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:52.370046 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c70ac9-6dac-4b7c-8749-b861ddfdb2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:52.372171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c70ac9-6dac-4b7c-8749-b861ddfdb2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:52.372445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c70ac9-6dac-4b7c-8749-b861ddfdb2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:52.372708 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c70ac9-6dac-4b7c-8749-b861ddfdb2d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:52.386406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c70ac9-6dac-4b7c-8749-b861ddfdb2d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:52.388244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c70ac9-6dac-4b7c-8749-b861ddfdb2d8 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-c7c70ac9-6dac-4b7c-8749-b861ddfdb2d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:52.414746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:52.415767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:52.416703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:52.416703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:52.416703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:52.416703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 264/513] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:52.418217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d0e4-aad8-420d-a263-bda4c7b948e8 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:52.418217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d0e4-aad8-420d-a263-bda4c7b948e8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:52.418217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d0e4-aad8-420d-a263-bda4c7b948e8 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:52.419648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 265/514] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:52.424976 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8d0e4-aad8-420d-a263-bda4c7b948e8 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:52.480860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:52.487325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5eec7-7f5d-4e9b-a4be-a1612022c1ee None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:52.530022 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abca804-e871-4ba5-bac1-b67d34c62517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:52.530652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abca804-e871-4ba5-bac1-b67d34c62517 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:52.530929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abca804-e871-4ba5-bac1-b67d34c62517 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:52.531200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abca804-e871-4ba5-bac1-b67d34c62517 None admin] PATH_INFO: `/v3/services/network` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:52.531947 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abca804-e871-4ba5-bac1-b67d34c62517 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:52.533375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abca804-e871-4ba5-bac1-b67d34c62517 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-7abca804-e871-4ba5-bac1-b67d34c62517 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:52.548163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:52.549184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:52.550029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 250/515] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:52 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:52.550838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb280ed-ac8c-4239-90ee-04c6d0e26405 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:52.550838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb280ed-ac8c-4239-90ee-04c6d0e26405 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:52.550838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb280ed-ac8c-4239-90ee-04c6d0e26405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:52.557409 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8d0e4-aad8-420d-a263-bda4c7b948e8 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:52.575309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-4bb280ed-ac8c-4239-90ee-04c6d0e26405 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:52.576725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-4bb280ed-ac8c-4239-90ee-04c6d0e26405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:52.593831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b9ff-d700-414c-8ea9-01ac861061a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:52.594682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b9ff-d700-414c-8ea9-01ac861061a2 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:52.595121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b9ff-d700-414c-8ea9-01ac861061a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:52.595541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b9ff-d700-414c-8ea9-01ac861061a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:52.596443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070b9ff-d700-414c-8ea9-01ac861061a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:52.596986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070b9ff-d700-414c-8ea9-01ac861061a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:52.598512 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070b9ff-d700-414c-8ea9-01ac861061a2 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:52.614101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 266/516] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:52.615121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b9ff-d700-414c-8ea9-01ac861061a2 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:52.622041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:52.662155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 251/517] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:52.662900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb280ed-ac8c-4239-90ee-04c6d0e26405 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:52.737964 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b9ff-d700-414c-8ea9-01ac861061a2 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:52.741983 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:52.782403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf5af7-2a16-4bec-8e3a-51761cb2ead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:52.783015 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf5af7-2a16-4bec-8e3a-51761cb2ead2 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:52.783280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf5af7-2a16-4bec-8e3a-51761cb2ead2 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:52.783548 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf5af7-2a16-4bec-8e3a-51761cb2ead2 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:52.784265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf5af7-2a16-4bec-8e3a-51761cb2ead2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:52.784574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf5af7-2a16-4bec-8e3a-51761cb2ead2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:52.785946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf5af7-2a16-4bec-8e3a-51761cb2ead2 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:52.791369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 267/518] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:42:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:52.796972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf5af7-2a16-4bec-8e3a-51761cb2ead2 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:52.804768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb280ed-ac8c-4239-90ee-04c6d0e26405 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:52.852561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bde829-4db2-463f-b319-d943a9f5d194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:52.853907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bde829-4db2-463f-b319-d943a9f5d194 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:52.854650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bde829-4db2-463f-b319-d943a9f5d194 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:52.855085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bde829-4db2-463f-b319-d943a9f5d194 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:52.864850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bde829-4db2-463f-b319-d943a9f5d194 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:52.867347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bde829-4db2-463f-b319-d943a9f5d194 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-a7bde829-4db2-463f-b319-d943a9f5d194 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:52.880534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:52.881746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:52.882886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:52.882886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:52.882886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:52.882886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:52.882886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:52.882886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:52.882886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:52.882886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:52.882886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:42:52.882886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:52.885788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 252/519] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Oct 14 17:42:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:52.893633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bde829-4db2-463f-b319-d943a9f5d194 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:52.940292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf5af7-2a16-4bec-8e3a-51761cb2ead2 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:52.957570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:52.986670 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe08fa-4222-49e4-8dcd-0cf830b71798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:52.987315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe08fa-4222-49e4-8dcd-0cf830b71798 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:52.987534 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe08fa-4222-49e4-8dcd-0cf830b71798 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:52.987800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe08fa-4222-49e4-8dcd-0cf830b71798 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:52.988456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe08fa-4222-49e4-8dcd-0cf830b71798 None admin] RBAC: Adding query filter params (type=network) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:52.988751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe08fa-4222-49e4-8dcd-0cf830b71798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:52.990113 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe08fa-4222-49e4-8dcd-0cf830b71798 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:52.996958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 268/520] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:42:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:53.049488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe08fa-4222-49e4-8dcd-0cf830b71798 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:53.101951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bde829-4db2-463f-b319-d943a9f5d194 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:53.125465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:53.147795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e7e2b-dcc2-482a-a6e7-bba3b3bba640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:53.148533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e7e2b-dcc2-482a-a6e7-bba3b3bba640 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:53.148753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e7e2b-dcc2-482a-a6e7-bba3b3bba640 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:53.149025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e7e2b-dcc2-482a-a6e7-bba3b3bba640 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:53.149770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e7e2b-dcc2-482a-a6e7-bba3b3bba640 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:53.150069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e7e2b-dcc2-482a-a6e7-bba3b3bba640 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:53.151542 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e7e2b-dcc2-482a-a6e7-bba3b3bba640 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:53.183693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 253/521] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Oct 14 17:42:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:53.189361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e7e2b-dcc2-482a-a6e7-bba3b3bba640 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:53.231717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe08fa-4222-49e4-8dcd-0cf830b71798 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:53.273629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f53b0a-1e5c-490b-aacf-d704e9767a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:53.274850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f53b0a-1e5c-490b-aacf-d704e9767a10 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:53.275123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f53b0a-1e5c-490b-aacf-d704e9767a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:53.275388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f53b0a-1e5c-490b-aacf-d704e9767a10 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:53.276282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f53b0a-1e5c-490b-aacf-d704e9767a10 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:53.278846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f53b0a-1e5c-490b-aacf-d704e9767a10 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-b2f53b0a-1e5c-490b-aacf-d704e9767a10 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:53.294137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:53.295169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:53.296067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 269/522] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Oct 14 17:42:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:53.301299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f53b0a-1e5c-490b-aacf-d704e9767a10 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:53.303855 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:53.326019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e7e2b-dcc2-482a-a6e7-bba3b3bba640 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:53.400768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216fe0a-93c9-4bcd-8112-53e707056abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:53.401785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216fe0a-93c9-4bcd-8112-53e707056abb None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:53.402271 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216fe0a-93c9-4bcd-8112-53e707056abb None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:53.402743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216fe0a-93c9-4bcd-8112-53e707056abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:53.428806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216fe0a-93c9-4bcd-8112-53e707056abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:53.430673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216fe0a-93c9-4bcd-8112-53e707056abb None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-b216fe0a-93c9-4bcd-8112-53e707056abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:53.453705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:53.454721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:53.455612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:53.455612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:53.455612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:53.455612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 254/523] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:53.464141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216fe0a-93c9-4bcd-8112-53e707056abb None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:53.464864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:53.474254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f53b0a-1e5c-490b-aacf-d704e9767a10 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:53.548136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9c379-e594-4fe8-b322-f3e293c10592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:53.548745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9c379-e594-4fe8-b322-f3e293c10592 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:53.549020 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9c379-e594-4fe8-b322-f3e293c10592 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:53.549287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9c379-e594-4fe8-b322-f3e293c10592 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:53.549919 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9c379-e594-4fe8-b322-f3e293c10592 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:53.550215 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9c379-e594-4fe8-b322-f3e293c10592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:53.553506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9c379-e594-4fe8-b322-f3e293c10592 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:53.562349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 270/524] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Oct 14 17:42:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:53.567945 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9c379-e594-4fe8-b322-f3e293c10592 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:53.628857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216fe0a-93c9-4bcd-8112-53e707056abb None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:53.665959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd39e5a-93ee-4194-b4cf-56a25b29bd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:53.666975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd39e5a-93ee-4194-b4cf-56a25b29bd47 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:53.666975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd39e5a-93ee-4194-b4cf-56a25b29bd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:53.667169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd39e5a-93ee-4194-b4cf-56a25b29bd47 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:53.667759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd39e5a-93ee-4194-b4cf-56a25b29bd47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:53.668087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd39e5a-93ee-4194-b4cf-56a25b29bd47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:53.669456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd39e5a-93ee-4194-b4cf-56a25b29bd47 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:53.683214 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 255/525] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:53.690794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd39e5a-93ee-4194-b4cf-56a25b29bd47 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:53.704258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:53.781383 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9c379-e594-4fe8-b322-f3e293c10592 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:53.845106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a895cc4-6e9a-46ed-aecf-d5941e7e389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:53.845759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a895cc4-6e9a-46ed-aecf-d5941e7e389a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:53.846035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a895cc4-6e9a-46ed-aecf-d5941e7e389a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:53.846298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a895cc4-6e9a-46ed-aecf-d5941e7e389a None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:53.846401 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd39e5a-93ee-4194-b4cf-56a25b29bd47 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:53.846925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a895cc4-6e9a-46ed-aecf-d5941e7e389a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:53.847216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a895cc4-6e9a-46ed-aecf-d5941e7e389a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:53.849776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a895cc4-6e9a-46ed-aecf-d5941e7e389a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:53.855936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 271/526] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Oct 14 17:42:53 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:53.857244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4286aa4-fd78-4dd5-a383-aed567873cf2 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:53.857523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4286aa4-fd78-4dd5-a383-aed567873cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:53.857791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4286aa4-fd78-4dd5-a383-aed567873cf2 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:53.859380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 272/527] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:53.869218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1cafe-e51e-4ff1-93d4-89ccfaae4e8c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:53.869578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1cafe-e51e-4ff1-93d4-89ccfaae4e8c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:53.869806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1cafe-e51e-4ff1-93d4-89ccfaae4e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:53.904119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f9b1cafe-e51e-4ff1-93d4-89ccfaae4e8c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:53.905332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f9b1cafe-e51e-4ff1-93d4-89ccfaae4e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:53.908835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653baae2-f57b-4c10-bd4b-3513985f0229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:53.909484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653baae2-f57b-4c10-bd4b-3513985f0229 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:53.909802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653baae2-f57b-4c10-bd4b-3513985f0229 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:53.910075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653baae2-f57b-4c10-bd4b-3513985f0229 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:53.918965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653baae2-f57b-4c10-bd4b-3513985f0229 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:53.921132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653baae2-f57b-4c10-bd4b-3513985f0229 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-653baae2-f57b-4c10-bd4b-3513985f0229 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:53.935986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:53.936933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:53.937822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:53.937822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:53.937822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:53.937822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:53.937822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:53.937822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:53.937822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:53.937822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:53.937822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 256/528] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:53.943408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653baae2-f57b-4c10-bd4b-3513985f0229 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:53.962085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:54.024052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:54.074469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653baae2-f57b-4c10-bd4b-3513985f0229 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:54.114724 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5652c-4a0f-42c8-ba49-63fe1671ab36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:54.115388 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5652c-4a0f-42c8-ba49-63fe1671ab36 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:54.115670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5652c-4a0f-42c8-ba49-63fe1671ab36 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:54.116523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5652c-4a0f-42c8-ba49-63fe1671ab36 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:54.117486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5652c-4a0f-42c8-ba49-63fe1671ab36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:54.118594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5652c-4a0f-42c8-ba49-63fe1671ab36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:54.120239 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5652c-4a0f-42c8-ba49-63fe1671ab36 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:54.128802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 273/529] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:54.135705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 257/530] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:54.141847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1cafe-e51e-4ff1-93d4-89ccfaae4e8c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:54.246076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1cafe-e51e-4ff1-93d4-89ccfaae4e8c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:54.282329 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03033a-137d-4660-b0cd-d6d1aab50c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:54.282993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03033a-137d-4660-b0cd-d6d1aab50c02 None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:54.283264 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03033a-137d-4660-b0cd-d6d1aab50c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:54.283535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03033a-137d-4660-b0cd-d6d1aab50c02 None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/e42968016de8470c933655e921fb3394/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:54.285821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03033a-137d-4660-b0cd-d6d1aab50c02 None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=e42968016de8470c933655e921fb3394, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:54.287315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03033a-137d-4660-b0cd-d6d1aab50c02 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:54.305144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 274/531] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:54 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/e42968016de8470c933655e921fb3394/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:54.508546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5652c-4a0f-42c8-ba49-63fe1671ab36 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:54.626749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5652c-4a0f-42c8-ba49-63fe1671ab36 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:54.657788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:54.672432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2864f29-bf9f-4611-a882-760752468e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:54.673063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864f29-bf9f-4611-a882-760752468e2a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:54.673335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864f29-bf9f-4611-a882-760752468e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:54.673605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864f29-bf9f-4611-a882-760752468e2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:54.692775 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2864f29-bf9f-4611-a882-760752468e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-d2864f29-bf9f-4611-a882-760752468e2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:54.698869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:54.699799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:54.700664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 258/532] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:42:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:54.706203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03033a-137d-4660-b0cd-d6d1aab50c02 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:54.736121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:54.867637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03033a-137d-4660-b0cd-d6d1aab50c02 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:54.906291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fd4e0-a792-4821-af28-f4b081ab8fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:54.906910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fd4e0-a792-4821-af28-f4b081ab8fcd None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:54.907184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fd4e0-a792-4821-af28-f4b081ab8fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:54.907450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fd4e0-a792-4821-af28-f4b081ab8fcd None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:54.908151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fd4e0-a792-4821-af28-f4b081ab8fcd None admin] RBAC: Adding query filter params (name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:54.908452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fd4e0-a792-4821-af28-f4b081ab8fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:54.909835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fd4e0-a792-4821-af28-f4b081ab8fcd None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:54.919147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 275/533] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:42:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:54.924801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2864f29-bf9f-4611-a882-760752468e2a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:55.014228 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c24038-051b-4981-86ff-4ee9dc4a8bcc None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:55.014562 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c24038-051b-4981-86ff-4ee9dc4a8bcc None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:55.014880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c24038-051b-4981-86ff-4ee9dc4a8bcc None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:55.016710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 276/534] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:55.022400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3c1fc-e2cb-439c-89ea-ea75948316c3 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:55.022731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3c1fc-e2cb-439c-89ea-ea75948316c3 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:55.023225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3c1fc-e2cb-439c-89ea-ea75948316c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:55.052449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-b2f3c1fc-e2cb-439c-89ea-ea75948316c3 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:55.054466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b2f3c1fc-e2cb-439c-89ea-ea75948316c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:55.116924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2864f29-bf9f-4611-a882-760752468e2a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:55.180066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 277/535] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:55.198563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027ce9-f7ae-495a-9a53-bcb1cbb6964e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:55.199199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027ce9-f7ae-495a-9a53-bcb1cbb6964e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:55.199470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027ce9-f7ae-495a-9a53-bcb1cbb6964e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:55.199737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027ce9-f7ae-495a-9a53-bcb1cbb6964e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:55.218058 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a027ce9-f7ae-495a-9a53-bcb1cbb6964e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:55.219606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a027ce9-f7ae-495a-9a53-bcb1cbb6964e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-5a027ce9-f7ae-495a-9a53-bcb1cbb6964e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:55.240385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:55.241425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:55.242302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:55.242302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:55.242302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:55.244614 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 259/536] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:55.249924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3c1fc-e2cb-439c-89ea-ea75948316c3 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:55.261472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:55.385086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3c1fc-e2cb-439c-89ea-ea75948316c3 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:55.437740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec514-05b9-4535-a5bd-4055edbd3073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:55.438268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec514-05b9-4535-a5bd-4055edbd3073 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:55.438543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec514-05b9-4535-a5bd-4055edbd3073 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:55.438809 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec514-05b9-4535-a5bd-4055edbd3073 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:55.439457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ec514-05b9-4535-a5bd-4055edbd3073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:55.439754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ec514-05b9-4535-a5bd-4055edbd3073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:55.441166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ec514-05b9-4535-a5bd-4055edbd3073 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:55.458376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 278/537] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:55.464369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027ce9-f7ae-495a-9a53-bcb1cbb6964e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:55.505518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:55.603981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec514-05b9-4535-a5bd-4055edbd3073 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:55.609789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027ce9-f7ae-495a-9a53-bcb1cbb6964e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:55.652959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34b431-0160-4df6-8639-193b76c8b1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:55.653584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34b431-0160-4df6-8639-193b76c8b1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:55.653857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34b431-0160-4df6-8639-193b76c8b1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:55.654127 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34b431-0160-4df6-8639-193b76c8b1d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:55.661669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34b431-0160-4df6-8639-193b76c8b1d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:55.663412 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34b431-0160-4df6-8639-193b76c8b1d9 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-eb34b431-0160-4df6-8639-193b76c8b1d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:55.679747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:55.680871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:55.682087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:55.682087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:55.682087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:55.682087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:55.682087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:55.682087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:55.682087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:55.682087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:55.682087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 17:42:55.682087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:55.683897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 260/538] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Oct 14 17:42:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:55.685183 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b78fe-a213-4336-bace-7fc7ccc72aa6 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:55.685589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b78fe-a213-4336-bace-7fc7ccc72aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:55.686041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b78fe-a213-4336-bace-7fc7ccc72aa6 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:55.687622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 261/539] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:55.695537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f4e23-02ef-4729-841f-4f1cbc070e3c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:55.695768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f4e23-02ef-4729-841f-4f1cbc070e3c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:55.696041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f4e23-02ef-4729-841f-4f1cbc070e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:55.701947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:55.727541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-d41f4e23-02ef-4729-841f-4f1cbc070e3c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:55.729292 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d41f4e23-02ef-4729-841f-4f1cbc070e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:55.765882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec514-05b9-4535-a5bd-4055edbd3073 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:55.839431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3130df-9300-4be6-ba06-c03eed0331d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:55.840335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3130df-9300-4be6-ba06-c03eed0331d9 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:55.841807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3130df-9300-4be6-ba06-c03eed0331d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:55.842236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3130df-9300-4be6-ba06-c03eed0331d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:55.870044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 262/540] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 17:42:55.870938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3130df-9300-4be6-ba06-c03eed0331d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:55.871354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f4e23-02ef-4729-841f-4f1cbc070e3c None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-db3130df-9300-4be6-ba06-c03eed0331d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:55.884196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:55.885249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:55.886182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 279/541] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:42:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:55.893160 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3130df-9300-4be6-ba06-c03eed0331d9 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:55.917854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:56.081932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f4e23-02ef-4729-841f-4f1cbc070e3c None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:56.129816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d666409-a60f-451f-b8a3-53da27fdc098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:56.130419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d666409-a60f-451f-b8a3-53da27fdc098 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:56.130692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d666409-a60f-451f-b8a3-53da27fdc098 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:56.130957 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d666409-a60f-451f-b8a3-53da27fdc098 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:56.131717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d666409-a60f-451f-b8a3-53da27fdc098 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:56.134291 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d666409-a60f-451f-b8a3-53da27fdc098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:56.135706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d666409-a60f-451f-b8a3-53da27fdc098 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:56.167642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 263/542] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Oct 14 17:42:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:56.175905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d666409-a60f-451f-b8a3-53da27fdc098 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:56.207712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3130df-9300-4be6-ba06-c03eed0331d9 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:56.291792 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:56.302616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee479b8-c693-4ff1-bc07-ec0664d7868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:56.304186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee479b8-c693-4ff1-bc07-ec0664d7868e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:56.304667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee479b8-c693-4ff1-bc07-ec0664d7868e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:56.305129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee479b8-c693-4ff1-bc07-ec0664d7868e None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:56.306007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee479b8-c693-4ff1-bc07-ec0664d7868e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:56.306522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee479b8-c693-4ff1-bc07-ec0664d7868e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:56.308243 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee479b8-c693-4ff1-bc07-ec0664d7868e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:56.324739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 280/543] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:56.325375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:56.326545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee479b8-c693-4ff1-bc07-ec0664d7868e None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:56.347391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d666409-a60f-451f-b8a3-53da27fdc098 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:56.390545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c5fcf-4098-4bb8-9794-328289607d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:56.391179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c5fcf-4098-4bb8-9794-328289607d53 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:56.391451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c5fcf-4098-4bb8-9794-328289607d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:56.391719 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c5fcf-4098-4bb8-9794-328289607d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:56.403824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c5fcf-4098-4bb8-9794-328289607d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:56.405902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c5fcf-4098-4bb8-9794-328289607d53 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-a29c5fcf-4098-4bb8-9794-328289607d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:56.421681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:56.422714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:56.423596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:56.423596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:56.423596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:56.423596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 264/544] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:56.424309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c5fcf-4098-4bb8-9794-328289607d53 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:56.452504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee479b8-c693-4ff1-bc07-ec0664d7868e None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:56.489653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a22aea-2008-46c5-937b-fb859e02c3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:56.490499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a22aea-2008-46c5-937b-fb859e02c3f0 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:56.491032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a22aea-2008-46c5-937b-fb859e02c3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:56.491970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a22aea-2008-46c5-937b-fb859e02c3f0 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:56.492848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a22aea-2008-46c5-937b-fb859e02c3f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:56.494442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a22aea-2008-46c5-937b-fb859e02c3f0 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:56.520099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 281/545] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:42:56 2022] POST /identity/v3/services => generated 240 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:42:56.521368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a22aea-2008-46c5-937b-fb859e02c3f0 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:56.535855 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c5fcf-4098-4bb8-9794-328289607d53 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:56.578298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:56.594119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce81d59-93e3-46fb-899f-3435a93fa57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:56.594995 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce81d59-93e3-46fb-899f-3435a93fa57d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:56.595385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce81d59-93e3-46fb-899f-3435a93fa57d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:56.603754 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce81d59-93e3-46fb-899f-3435a93fa57d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:56.621636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce81d59-93e3-46fb-899f-3435a93fa57d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:56.624532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce81d59-93e3-46fb-899f-3435a93fa57d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:56.637485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-6ce81d59-93e3-46fb-899f-3435a93fa57d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:56.654119 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:56.655144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:56.656072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:56.656072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:56.656072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:56.663988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 265/546] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:56.672084 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce81d59-93e3-46fb-899f-3435a93fa57d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:56.748998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a22aea-2008-46c5-937b-fb859e02c3f0 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:56.795585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c435c0-86d2-422c-a1d9-764a6391486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:56.796258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c435c0-86d2-422c-a1d9-764a6391486d None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:56.796514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c435c0-86d2-422c-a1d9-764a6391486d None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:56.796784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c435c0-86d2-422c-a1d9-764a6391486d None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:56.797444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c435c0-86d2-422c-a1d9-764a6391486d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:56.797742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c435c0-86d2-422c-a1d9-764a6391486d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:56.799203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c435c0-86d2-422c-a1d9-764a6391486d None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:56.814167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 282/547] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:56.815101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916e711-3c9d-4f33-85c0-67fbabb842d7 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:56.815385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916e711-3c9d-4f33-85c0-67fbabb842d7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:56.815653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916e711-3c9d-4f33-85c0-67fbabb842d7 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:56.816920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 283/548] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:56.821421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916e711-3c9d-4f33-85c0-67fbabb842d7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:56.824558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce81d59-93e3-46fb-899f-3435a93fa57d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:56.864429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82394e-e30f-4719-a497-e820ce402ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:56.865318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82394e-e30f-4719-a497-e820ce402ddd None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:56.865783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82394e-e30f-4719-a497-e820ce402ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:56.866777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82394e-e30f-4719-a497-e820ce402ddd None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:56.867607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82394e-e30f-4719-a497-e820ce402ddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:56.868183 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82394e-e30f-4719-a497-e820ce402ddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:56.869866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82394e-e30f-4719-a497-e820ce402ddd None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:56.883110 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 266/549] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:56.884776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76a335-fd3d-4ff3-a38a-4ef60ddc5e28 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:56.885362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76a335-fd3d-4ff3-a38a-4ef60ddc5e28 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:56.885916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76a335-fd3d-4ff3-a38a-4ef60ddc5e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:56.905710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-eb76a335-fd3d-4ff3-a38a-4ef60ddc5e28 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:56.907409 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-eb76a335-fd3d-4ff3-a38a-4ef60ddc5e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:56.949152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:56.953629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916e711-3c9d-4f33-85c0-67fbabb842d7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:56.964856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:56.992202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 267/550] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:56 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:56.992744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b637f5-0583-429c-93bd-1e9bd4f5a91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:56.993269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76a335-fd3d-4ff3-a38a-4ef60ddc5e28 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:56.993890 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b637f5-0583-429c-93bd-1e9bd4f5a91a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:56.994168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b637f5-0583-429c-93bd-1e9bd4f5a91a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:56.994432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b637f5-0583-429c-93bd-1e9bd4f5a91a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.005344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b637f5-0583-429c-93bd-1e9bd4f5a91a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:57.007442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b637f5-0583-429c-93bd-1e9bd4f5a91a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-a0b637f5-0583-429c-93bd-1e9bd4f5a91a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:57.019158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:57.020146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:57.021048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:57.021048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:57.021048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:57.021048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:57.021048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:57.021048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:57.021048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:57.021048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:57.025153 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 284/551] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:57.026451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718b317-0f2c-49a6-85fc-5663c764e30f None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:57.026732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718b317-0f2c-49a6-85fc-5663c764e30f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:57.027000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718b317-0f2c-49a6-85fc-5663c764e30f None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.032861 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 285/552] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:42:57.033793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718b317-0f2c-49a6-85fc-5663c764e30f None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:57.124150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76a335-fd3d-4ff3-a38a-4ef60ddc5e28 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:57.160436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718b317-0f2c-49a6-85fc-5663c764e30f None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:57.167381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e5559-02f4-485d-ae35-5b998c892dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:57.168327 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5559-02f4-485d-ae35-5b998c892dfd None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:57.168705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5559-02f4-485d-ae35-5b998c892dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:57.169081 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5559-02f4-485d-ae35-5b998c892dfd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.179155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e5559-02f4-485d-ae35-5b998c892dfd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:57.181087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e5559-02f4-485d-ae35-5b998c892dfd None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-e90e5559-02f4-485d-ae35-5b998c892dfd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:57.194783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:57.196213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:57.197371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:57.197371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:57.197371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:57.197371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:57.197371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:57.197371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:57.197371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:57.197371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:57.197371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 17:42:57.197371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:57.199205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 268/553] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:42:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:57.200946 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6dc98-d00b-4876-82ea-90dba9a7f236 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:57.201325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6dc98-d00b-4876-82ea-90dba9a7f236 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:57.201790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6dc98-d00b-4876-82ea-90dba9a7f236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.207711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ea072-affb-4c14-a549-93ef3aaead20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:57.208785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ea072-affb-4c14-a549-93ef3aaead20 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:57.210887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ea072-affb-4c14-a549-93ef3aaead20 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:57.211310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ea072-affb-4c14-a549-93ef3aaead20 None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.212188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ea072-affb-4c14-a549-93ef3aaead20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:57.212651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ea072-affb-4c14-a549-93ef3aaead20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:57.214225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ea072-affb-4c14-a549-93ef3aaead20 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:57.221892 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-7ac6dc98-d00b-4876-82ea-90dba9a7f236 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:57.223565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7ac6dc98-d00b-4876-82ea-90dba9a7f236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:57.229176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 286/554] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:57.230268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ea072-affb-4c14-a549-93ef3aaead20 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:57.258846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:57.266065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:57.310874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 269/555] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:57 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 17:42:57.311758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6dc98-d00b-4876-82ea-90dba9a7f236 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:57.365040 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ea072-affb-4c14-a549-93ef3aaead20 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:57.410848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1deb79-f3ea-4907-9301-53ee41d00db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:57.411381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1deb79-f3ea-4907-9301-53ee41d00db5 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:57.411658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1deb79-f3ea-4907-9301-53ee41d00db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:57.411949 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1deb79-f3ea-4907-9301-53ee41d00db5 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.412691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1deb79-f3ea-4907-9301-53ee41d00db5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:57.413664 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1deb79-f3ea-4907-9301-53ee41d00db5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:57.415134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1deb79-f3ea-4907-9301-53ee41d00db5 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:57.429228 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6dc98-d00b-4876-82ea-90dba9a7f236 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:57.439899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 287/556] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Oct 14 17:42:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:57.440856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1deb79-f3ea-4907-9301-53ee41d00db5 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:57.470558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ad37e-c374-4557-9e72-fc0c87a25a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:57.471633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ad37e-c374-4557-9e72-fc0c87a25a77 None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:57.472262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ad37e-c374-4557-9e72-fc0c87a25a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:57.472823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ad37e-c374-4557-9e72-fc0c87a25a77 None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/d549c67a35a94c65a5cc2a3f3ec83474/roles/e1a3bea83ac349acb6d65b6afb5441d6` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.491227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ad37e-c374-4557-9e72-fc0c87a25a77 None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=d549c67a35a94c65a5cc2a3f3ec83474, role_id=e1a3bea83ac349acb6d65b6afb5441d6)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:57.493589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ad37e-c374-4557-9e72-fc0c87a25a77 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:57.511759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 270/557] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:57 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/d549c67a35a94c65a5cc2a3f3ec83474/roles/e1a3bea83ac349acb6d65b6afb5441d6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:42:57.512976 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ad37e-c374-4557-9e72-fc0c87a25a77 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:57.536953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:57.595703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:57.619954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1deb79-f3ea-4907-9301-53ee41d00db5 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:57.648306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ad37e-c374-4557-9e72-fc0c87a25a77 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:57.670223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1249b-3da9-440e-b9c0-ab1e56bf7cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:57.670762 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1249b-3da9-440e-b9c0-ab1e56bf7cdf None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:57.671035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1249b-3da9-440e-b9c0-ab1e56bf7cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:57.671303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1249b-3da9-440e-b9c0-ab1e56bf7cdf None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.672364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1249b-3da9-440e-b9c0-ab1e56bf7cdf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:57.673652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1249b-3da9-440e-b9c0-ab1e56bf7cdf None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:57.703986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 288/558] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:42:57 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:42:57.705355 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1249b-3da9-440e-b9c0-ab1e56bf7cdf None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:57.707188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa9ca2-e732-4dd5-9f9b-cdd4c20f2979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:57.708613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa9ca2-e732-4dd5-9f9b-cdd4c20f2979 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:57.709316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa9ca2-e732-4dd5-9f9b-cdd4c20f2979 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:57.711294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa9ca2-e732-4dd5-9f9b-cdd4c20f2979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.738159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa9ca2-e732-4dd5-9f9b-cdd4c20f2979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:57.739738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa9ca2-e732-4dd5-9f9b-cdd4c20f2979 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-48aa9ca2-e732-4dd5-9f9b-cdd4c20f2979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:57.776539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:57.777598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:57.778484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:57.778484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:57.778484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:57.778484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 271/559] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:57.784918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa9ca2-e732-4dd5-9f9b-cdd4c20f2979 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:57.856052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:57.870497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1249b-3da9-440e-b9c0-ab1e56bf7cdf None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:57.907306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa9ca2-e732-4dd5-9f9b-cdd4c20f2979 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:57.913110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eedd0c-bb10-46c7-9233-1b3f0efdb854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:57.914957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eedd0c-bb10-46c7-9233-1b3f0efdb854 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:57.915521 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eedd0c-bb10-46c7-9233-1b3f0efdb854 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:57.915957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eedd0c-bb10-46c7-9233-1b3f0efdb854 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.930430 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eedd0c-bb10-46c7-9233-1b3f0efdb854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-39eedd0c-bb10-46c7-9233-1b3f0efdb854 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:42:57.935978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:57.937143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:42:57.938151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:57.940449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 289/560] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:42:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:57.947157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eedd0c-bb10-46c7-9233-1b3f0efdb854 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:57.950312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b962e6-bac9-4dd7-9ce4-683966dd17e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:57.951094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b962e6-bac9-4dd7-9ce4-683966dd17e8 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:57.952237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b962e6-bac9-4dd7-9ce4-683966dd17e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:57.952237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b962e6-bac9-4dd7-9ce4-683966dd17e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:57.952706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b962e6-bac9-4dd7-9ce4-683966dd17e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:57.953102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b962e6-bac9-4dd7-9ce4-683966dd17e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:57.954575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b962e6-bac9-4dd7-9ce4-683966dd17e8 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:57.959162 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:57.972608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 272/561] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:42:57.978966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b962e6-bac9-4dd7-9ce4-683966dd17e8 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:58.083529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eedd0c-bb10-46c7-9233-1b3f0efdb854 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:58.113547 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b962e6-bac9-4dd7-9ce4-683966dd17e8 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:58.131209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dcabe-ab5d-4907-8f3f-3af777f92613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:58.132168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dcabe-ab5d-4907-8f3f-3af777f92613 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:58.132629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dcabe-ab5d-4907-8f3f-3af777f92613 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:58.133043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dcabe-ab5d-4907-8f3f-3af777f92613 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:58.133877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dcabe-ab5d-4907-8f3f-3af777f92613 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:58.134322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dcabe-ab5d-4907-8f3f-3af777f92613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:58.135839 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dcabe-ab5d-4907-8f3f-3af777f92613 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:58.140271 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:58.146936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 290/562] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:58.154428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dcabe-ab5d-4907-8f3f-3af777f92613 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:58.157526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16040c3a-3781-4053-b321-6c85b54b5f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:58.158265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16040c3a-3781-4053-b321-6c85b54b5f39 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:58.158643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16040c3a-3781-4053-b321-6c85b54b5f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:58.159018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16040c3a-3781-4053-b321-6c85b54b5f39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:58.171917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16040c3a-3781-4053-b321-6c85b54b5f39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:58.171917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16040c3a-3781-4053-b321-6c85b54b5f39 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-16040c3a-3781-4053-b321-6c85b54b5f39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:42:58.185455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:58.186556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:58.187651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:42:58.187651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:42:58.187651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:42:58.187651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:42:58.187651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:42:58.187651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:42:58.187651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:42:58.187651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:58.189561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 273/563] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:58.196596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16040c3a-3781-4053-b321-6c85b54b5f39 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:58.289473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:58.339773 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dcabe-ab5d-4907-8f3f-3af777f92613 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:58.415047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16040c3a-3781-4053-b321-6c85b54b5f39 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:58.416084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0dc9d-ab37-4ba0-91dd-1d37a1ded6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:58.416803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0dc9d-ab37-4ba0-91dd-1d37a1ded6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:58.417185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0dc9d-ab37-4ba0-91dd-1d37a1ded6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:58.417560 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0dc9d-ab37-4ba0-91dd-1d37a1ded6c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:58.442104 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0dc9d-ab37-4ba0-91dd-1d37a1ded6c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:58.443527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0dc9d-ab37-4ba0-91dd-1d37a1ded6c7 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:58.466265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdd329-5ef4-402c-97b3-924c4ef3e05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:58.467033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdd329-5ef4-402c-97b3-924c4ef3e05f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:58.467415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdd329-5ef4-402c-97b3-924c4ef3e05f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:58.467792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdd329-5ef4-402c-97b3-924c4ef3e05f None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:58.468622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdd329-5ef4-402c-97b3-924c4ef3e05f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:58.469025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdd329-5ef4-402c-97b3-924c4ef3e05f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:58.470612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdd329-5ef4-402c-97b3-924c4ef3e05f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-6be0dc9d-ab37-4ba0-91dd-1d37a1ded6c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:58.471905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:58.473024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:58.474141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:58.474141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:58.474141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:58.481168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 291/564] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:58.486675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0dc9d-ab37-4ba0-91dd-1d37a1ded6c7 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:58.490074 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 274/565] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:42:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:58.495699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdd329-5ef4-402c-97b3-924c4ef3e05f None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:58.618687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:58.627807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0dc9d-ab37-4ba0-91dd-1d37a1ded6c7 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:58.660318 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:58.682828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc8eb8-d229-4708-804f-64606d5627b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:58.684584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc8eb8-d229-4708-804f-64606d5627b8 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:58.686049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc8eb8-d229-4708-804f-64606d5627b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:58.686396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc8eb8-d229-4708-804f-64606d5627b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:58.689498 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc8eb8-d229-4708-804f-64606d5627b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:58.689877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc8eb8-d229-4708-804f-64606d5627b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:58.691367 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc8eb8-d229-4708-804f-64606d5627b8 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:58.727275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 292/566] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:58.745166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc8eb8-d229-4708-804f-64606d5627b8 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:58.755808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdd329-5ef4-402c-97b3-924c4ef3e05f None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:58.837737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279834ee-8c07-47bc-87bd-91dcb65f7091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:58.837737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279834ee-8c07-47bc-87bd-91dcb65f7091 None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:58.837737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279834ee-8c07-47bc-87bd-91dcb65f7091 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:58.837737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279834ee-8c07-47bc-87bd-91dcb65f7091 None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/9ec9c3fcf7964640a5eeabc01cd97617/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:58.838203 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279834ee-8c07-47bc-87bd-91dcb65f7091 None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=9ec9c3fcf7964640a5eeabc01cd97617, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:58.838203 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279834ee-8c07-47bc-87bd-91dcb65f7091 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:58.868146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 275/567] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:42:58 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/9ec9c3fcf7964640a5eeabc01cd97617/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:42:58.903217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc8eb8-d229-4708-804f-64606d5627b8 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:58.949386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23d68e-7619-44fc-b4ad-3ea7817adbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:58.950075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23d68e-7619-44fc-b4ad-3ea7817adbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:58.950352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23d68e-7619-44fc-b4ad-3ea7817adbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:58.950619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23d68e-7619-44fc-b4ad-3ea7817adbc3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:58.958407 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23d68e-7619-44fc-b4ad-3ea7817adbc3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:58.960008 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23d68e-7619-44fc-b4ad-3ea7817adbc3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-fb23d68e-7619-44fc-b4ad-3ea7817adbc3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:42:58.973540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:58.974569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:58.977540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 293/568] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Oct 14 17:42:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:42:58.981561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279834ee-8c07-47bc-87bd-91dcb65f7091 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:59.009369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a9090-31c4-4538-8644-40ea3d53314f None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:59.009737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a9090-31c4-4538-8644-40ea3d53314f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:59.009910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a9090-31c4-4538-8644-40ea3d53314f None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:59.011413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 294/569] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:42:59.019381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ce78d-2c46-4bce-8f7c-b9e8963d7734 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:59.020128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ce78d-2c46-4bce-8f7c-b9e8963d7734 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:59.020646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ce78d-2c46-4bce-8f7c-b9e8963d7734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:59.054894 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ab7ce78d-2c46-4bce-8f7c-b9e8963d7734 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:42:59.057152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ab7ce78d-2c46-4bce-8f7c-b9e8963d7734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:42:59.128542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:59.154485 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:59.158147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279834ee-8c07-47bc-87bd-91dcb65f7091 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:59.174225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 295/570] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:42:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:42:59.203152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab05de5-694d-4b68-9fd2-0387b62571b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:59.203779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab05de5-694d-4b68-9fd2-0387b62571b6 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:59.204103 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab05de5-694d-4b68-9fd2-0387b62571b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:59.204346 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab05de5-694d-4b68-9fd2-0387b62571b6 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:59.205076 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab05de5-694d-4b68-9fd2-0387b62571b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:42:59.205390 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab05de5-694d-4b68-9fd2-0387b62571b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:59.206812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab05de5-694d-4b68-9fd2-0387b62571b6 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:59.225358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 276/571] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Oct 14 17:42:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:42:59.231625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ce78d-2c46-4bce-8f7c-b9e8963d7734 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:59.363917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ce78d-2c46-4bce-8f7c-b9e8963d7734 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:59.415523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dd58a-c9a1-46a2-b20e-0ec5ff3cb23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:59.416162 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dd58a-c9a1-46a2-b20e-0ec5ff3cb23e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:59.416503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dd58a-c9a1-46a2-b20e-0ec5ff3cb23e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:59.416777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dd58a-c9a1-46a2-b20e-0ec5ff3cb23e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:59.466491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:59.506124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dd58a-c9a1-46a2-b20e-0ec5ff3cb23e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:59.508160 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dd58a-c9a1-46a2-b20e-0ec5ff3cb23e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:59.531431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab05de5-694d-4b68-9fd2-0387b62571b6 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:59.680945 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 296/572] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:42:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:42:59.688550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dd58a-c9a1-46a2-b20e-0ec5ff3cb23e None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:59.724868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab05de5-694d-4b68-9fd2-0387b62571b6 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:59.794887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818332d3-2fb6-4aed-aeac-980bd3d459aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:59.795604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818332d3-2fb6-4aed-aeac-980bd3d459aa None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:59.795865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818332d3-2fb6-4aed-aeac-980bd3d459aa None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:59.796147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818332d3-2fb6-4aed-aeac-980bd3d459aa None admin] PATH_INFO: `/v3/services/network` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:59.796723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818332d3-2fb6-4aed-aeac-980bd3d459aa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:59.798075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818332d3-2fb6-4aed-aeac-980bd3d459aa None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-818332d3-2fb6-4aed-aeac-980bd3d459aa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:59.810781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:59.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:42:59.812762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 277/573] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:42:59 2022] GET /identity/v3/services/network => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:59.820924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818332d3-2fb6-4aed-aeac-980bd3d459aa None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:42:59.821440 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:59.870474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:42:59.874958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dd58a-c9a1-46a2-b20e-0ec5ff3cb23e None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:42:59.942478 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97367b-01ea-47dd-a9e9-ae9c7685205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:42:59.943540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97367b-01ea-47dd-a9e9-ae9c7685205b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:42:59.944060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97367b-01ea-47dd-a9e9-ae9c7685205b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:42:59.944544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97367b-01ea-47dd-a9e9-ae9c7685205b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:42:59.963404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97367b-01ea-47dd-a9e9-ae9c7685205b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:42:59.964913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97367b-01ea-47dd-a9e9-ae9c7685205b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-1e97367b-01ea-47dd-a9e9-ae9c7685205b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:42:59.987540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:42:59.988603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:42:59.989545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:42:59.989545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:42:59.989545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:42:59.989545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 297/574] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:42:59.995408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97367b-01ea-47dd-a9e9-ae9c7685205b None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:00.017018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818332d3-2fb6-4aed-aeac-980bd3d459aa None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:00.054792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c45310-2eb4-431e-8054-ec45b6e8af4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:00.055322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c45310-2eb4-431e-8054-ec45b6e8af4b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.055597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c45310-2eb4-431e-8054-ec45b6e8af4b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.055863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c45310-2eb4-431e-8054-ec45b6e8af4b None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.056508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c45310-2eb4-431e-8054-ec45b6e8af4b None admin] RBAC: Adding query filter params (name=network) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:00.057535 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c45310-2eb4-431e-8054-ec45b6e8af4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:00.058170 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c45310-2eb4-431e-8054-ec45b6e8af4b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:00.063254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 278/575] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:42:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:00.074089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c45310-2eb4-431e-8054-ec45b6e8af4b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:00.209087 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97367b-01ea-47dd-a9e9-ae9c7685205b None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:00.245105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:00.249324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:00.262898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c45310-2eb4-431e-8054-ec45b6e8af4b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:00.265145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fc9c3-212b-40bc-88fa-de1436f81a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:00.265754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fc9c3-212b-40bc-88fa-de1436f81a0c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.266027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fc9c3-212b-40bc-88fa-de1436f81a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.266291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fc9c3-212b-40bc-88fa-de1436f81a0c None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.267849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fc9c3-212b-40bc-88fa-de1436f81a0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:00.268184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fc9c3-212b-40bc-88fa-de1436f81a0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:00.269624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fc9c3-212b-40bc-88fa-de1436f81a0c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:00.284437 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 298/576] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:00.285958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8398519-3d12-4ed4-b922-58bd4aa75480 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.286495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8398519-3d12-4ed4-b922-58bd4aa75480 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.287074 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8398519-3d12-4ed4-b922-58bd4aa75480 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.289016 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 299/577] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:00.289949 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8398519-3d12-4ed4-b922-58bd4aa75480 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:00.305723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a252e-8d95-418a-9be6-dab96b89d912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:00.306256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a252e-8d95-418a-9be6-dab96b89d912 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.306527 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a252e-8d95-418a-9be6-dab96b89d912 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.306856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a252e-8d95-418a-9be6-dab96b89d912 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.307470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a252e-8d95-418a-9be6-dab96b89d912 None admin] RBAC: Adding query filter params (type=network) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:00.307763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a252e-8d95-418a-9be6-dab96b89d912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:00.309169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a252e-8d95-418a-9be6-dab96b89d912 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:00.319912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 279/578] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:43:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:00.319912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a11018-e5e0-4a04-8b21-3798d36d1494 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.319912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a11018-e5e0-4a04-8b21-3798d36d1494 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.319912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a11018-e5e0-4a04-8b21-3798d36d1494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.339573 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-b5a11018-e5e0-4a04-8b21-3798d36d1494 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:00.340779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b5a11018-e5e0-4a04-8b21-3798d36d1494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:00.422746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 280/579] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:00.423998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a11018-e5e0-4a04-8b21-3798d36d1494 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:00.430461 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8398519-3d12-4ed4-b922-58bd4aa75480 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:00.503582 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0c1fe-2bd4-4ed9-a801-9e0af79ca417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:00.504641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0c1fe-2bd4-4ed9-a801-9e0af79ca417 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.505272 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0c1fe-2bd4-4ed9-a801-9e0af79ca417 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.506270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0c1fe-2bd4-4ed9-a801-9e0af79ca417 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.521133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0c1fe-2bd4-4ed9-a801-9e0af79ca417 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:00.524969 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0c1fe-2bd4-4ed9-a801-9e0af79ca417 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-66d0c1fe-2bd4-4ed9-a801-9e0af79ca417 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:43:00.549592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:00.550576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:00.551698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:43:00.551698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:43:00.551698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:43:00.551698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:43:00.551698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:43:00.551698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:43:00.551698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:00.551698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:00.557884 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 300/580] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:00.560858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0c1fe-2bd4-4ed9-a801-9e0af79ca417 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:00.603653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:00.644366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:00.645979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a11018-e5e0-4a04-8b21-3798d36d1494 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:00.687240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4284b45-d0f8-4808-b2c7-4c660a9c2443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:00.688381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4284b45-d0f8-4808-b2c7-4c660a9c2443 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.688661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4284b45-d0f8-4808-b2c7-4c660a9c2443 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.688930 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4284b45-d0f8-4808-b2c7-4c660a9c2443 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.689625 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4284b45-d0f8-4808-b2c7-4c660a9c2443 None admin] RBAC: Adding query filter params (interface=public, service_id=fe4f86dc831945fb9beba88139dee824, region_id=RegionOne) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:00.689923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4284b45-d0f8-4808-b2c7-4c660a9c2443 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:00.691723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4284b45-d0f8-4808-b2c7-4c660a9c2443 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:00.703618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 281/581] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Oct 14 17:43:00 2022] GET /identity/v3/endpoints?service_id=fe4f86dc831945fb9beba88139dee824&interface=public®ion_id=RegionOne => generated 194 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:00.708251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fb1ea-cb8c-4821-93cd-5578ebfe4d3e None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.708251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fb1ea-cb8c-4821-93cd-5578ebfe4d3e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.708251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fb1ea-cb8c-4821-93cd-5578ebfe4d3e None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.712055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 282/582] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:00.713855 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fb1ea-cb8c-4821-93cd-5578ebfe4d3e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:00.725592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0c1fe-2bd4-4ed9-a801-9e0af79ca417 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:00.762670 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28d5c7-0a6f-431a-a9fa-ed1804a797cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:00.763666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28d5c7-0a6f-431a-a9fa-ed1804a797cd None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.764043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28d5c7-0a6f-431a-a9fa-ed1804a797cd None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.764380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28d5c7-0a6f-431a-a9fa-ed1804a797cd None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.765129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28d5c7-0a6f-431a-a9fa-ed1804a797cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:00.765498 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28d5c7-0a6f-431a-a9fa-ed1804a797cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:00.766985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28d5c7-0a6f-431a-a9fa-ed1804a797cd None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:00.783216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 301/583] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:43:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:00.783216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc290b0-62db-49e3-be3f-cb07b5d0dd09 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.783216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc290b0-62db-49e3-be3f-cb07b5d0dd09 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.783216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc290b0-62db-49e3-be3f-cb07b5d0dd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.799006 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-5fc290b0-62db-49e3-be3f-cb07b5d0dd09 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:00.800317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5fc290b0-62db-49e3-be3f-cb07b5d0dd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:00.840067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fb1ea-cb8c-4821-93cd-5578ebfe4d3e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:00.871486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 302/584] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:00.872583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc290b0-62db-49e3-be3f-cb07b5d0dd09 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:00.889657 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ddc60-06f4-4ceb-b881-c7af7e5448be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:00.890270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ddc60-06f4-4ceb-b881-c7af7e5448be None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:00.890547 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ddc60-06f4-4ceb-b881-c7af7e5448be None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:00.890910 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ddc60-06f4-4ceb-b881-c7af7e5448be None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:00.891544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ddc60-06f4-4ceb-b881-c7af7e5448be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:00.893079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ddc60-06f4-4ceb-b881-c7af7e5448be None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:00.898665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 283/585] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Oct 14 17:43:00 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:00.899659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ddc60-06f4-4ceb-b881-c7af7e5448be None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:00.916491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:00.926245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:01.056654 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc290b0-62db-49e3-be3f-cb07b5d0dd09 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:01.112487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ddc60-06f4-4ceb-b881-c7af7e5448be None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:01.126769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205675-12dc-476d-a667-5e2cc838f501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:01.128202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205675-12dc-476d-a667-5e2cc838f501 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:01.128792 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205675-12dc-476d-a667-5e2cc838f501 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:01.129245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205675-12dc-476d-a667-5e2cc838f501 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:01.147034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9205675-12dc-476d-a667-5e2cc838f501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-d9205675-12dc-476d-a667-5e2cc838f501 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:43:01.155663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:01.156712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:43:01.157597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:01.160283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 303/586] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:43:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:01.164951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa7452-2d70-481b-b0b5-230f4dc9c434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:01.165198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa7452-2d70-481b-b0b5-230f4dc9c434 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:01.165446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa7452-2d70-481b-b0b5-230f4dc9c434 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:01.165710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa7452-2d70-481b-b0b5-230f4dc9c434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:01.166427 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa7452-2d70-481b-b0b5-230f4dc9c434 None admin] RBAC: Adding query filter params (role.id=45b67ba9922c4b16baae3930d3de1b6c, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=e42968016de8470c933655e921fb3394) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:01.166725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa7452-2d70-481b-b0b5-230f4dc9c434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:01.168347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa7452-2d70-481b-b0b5-230f4dc9c434 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:01.174757 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205675-12dc-476d-a667-5e2cc838f501 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:01.175623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 284/587] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:43:00 2022] GET /identity/v3/role_assignments?user.id=e42968016de8470c933655e921fb3394&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=45b67ba9922c4b16baae3930d3de1b6c => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:01.212205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa7452-2d70-481b-b0b5-230f4dc9c434 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:01.311513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:01.319333 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:01.327451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205675-12dc-476d-a667-5e2cc838f501 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:01.381606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa7452-2d70-481b-b0b5-230f4dc9c434 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:01.400900 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992395c-c6e0-4a87-9cca-748f278d121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:01.401509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992395c-c6e0-4a87-9cca-748f278d121c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:01.401781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992395c-c6e0-4a87-9cca-748f278d121c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:01.402047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992395c-c6e0-4a87-9cca-748f278d121c None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:01.402715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992395c-c6e0-4a87-9cca-748f278d121c None admin] RBAC: Adding query filter params (name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:01.403009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992395c-c6e0-4a87-9cca-748f278d121c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:01.404413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992395c-c6e0-4a87-9cca-748f278d121c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:01.418175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 304/588] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:43:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:01.425634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992395c-c6e0-4a87-9cca-748f278d121c None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:01.442922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623b48f-2a45-4b71-beb6-554890bb7da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:01.443563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623b48f-2a45-4b71-beb6-554890bb7da9 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:01.443835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623b48f-2a45-4b71-beb6-554890bb7da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:01.444261 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623b48f-2a45-4b71-beb6-554890bb7da9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:01.445447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623b48f-2a45-4b71-beb6-554890bb7da9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:01.447689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623b48f-2a45-4b71-beb6-554890bb7da9 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-2623b48f-2a45-4b71-beb6-554890bb7da9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:01.462364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:01.463434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:01.464329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 285/589] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Oct 14 17:43:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:01.471531 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623b48f-2a45-4b71-beb6-554890bb7da9 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:01.557867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992395c-c6e0-4a87-9cca-748f278d121c None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:01.605120 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623b48f-2a45-4b71-beb6-554890bb7da9 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:01.609010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0df1d-2bce-44ae-be83-f54d72f2437e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:01.609647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0df1d-2bce-44ae-be83-f54d72f2437e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:01.609920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0df1d-2bce-44ae-be83-f54d72f2437e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:01.610184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0df1d-2bce-44ae-be83-f54d72f2437e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:01.625403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0df1d-2bce-44ae-be83-f54d72f2437e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:01.626864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0df1d-2bce-44ae-be83-f54d72f2437e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:01.643281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86df5fc-eb49-4634-aecc-689a45ccb95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:01.643810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df5fc-eb49-4634-aecc-689a45ccb95f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:01.644110 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df5fc-eb49-4634-aecc-689a45ccb95f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:01.644375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df5fc-eb49-4634-aecc-689a45ccb95f None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:01.644993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86df5fc-eb49-4634-aecc-689a45ccb95f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:01.645283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86df5fc-eb49-4634-aecc-689a45ccb95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-f0e0df1d-2bce-44ae-be83-f54d72f2437e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:01.645914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:01.648605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:01.650770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:01.650770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:01.650770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:01.651148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86df5fc-eb49-4634-aecc-689a45ccb95f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:01.651304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 305/590] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:01.651304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2d52-2bb8-4a93-9925-dfd6e318e4c0 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:01.651304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2d52-2bb8-4a93-9925-dfd6e318e4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:01.651304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2d52-2bb8-4a93-9925-dfd6e318e4c0 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:01.651304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 306/591] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:01.656867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:01.656997 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d2d52-2bb8-4a93-9925-dfd6e318e4c0 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:01.660974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 286/592] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Oct 14 17:43:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:01.662436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94180d10-8039-4002-b61d-767f02054661 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:01.662727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94180d10-8039-4002-b61d-767f02054661 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:01.663580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94180d10-8039-4002-b61d-767f02054661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:01.680389 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:01.682837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-94180d10-8039-4002-b61d-767f02054661 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:01.684255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-94180d10-8039-4002-b61d-767f02054661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:01.767903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 287/593] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:01.767903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94180d10-8039-4002-b61d-767f02054661 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:01.832190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d2d52-2bb8-4a93-9925-dfd6e318e4c0 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:01.868299 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94180d10-8039-4002-b61d-767f02054661 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:01.884615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f6ef8-6c8e-49f6-bd3d-19795681c50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:01.884615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f6ef8-6c8e-49f6-bd3d-19795681c50c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:01.884615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f6ef8-6c8e-49f6-bd3d-19795681c50c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:01.884615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f6ef8-6c8e-49f6-bd3d-19795681c50c None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:01.885300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f6ef8-6c8e-49f6-bd3d-19795681c50c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:01.886836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f6ef8-6c8e-49f6-bd3d-19795681c50c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:01.887001 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f6ef8-6c8e-49f6-bd3d-19795681c50c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:01.908083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 307/594] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:01.914142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f6ef8-6c8e-49f6-bd3d-19795681c50c None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:01.935675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:01.946997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef1718-34b1-46bc-ad62-fb4487a430fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:01.947621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef1718-34b1-46bc-ad62-fb4487a430fe None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:01.947925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef1718-34b1-46bc-ad62-fb4487a430fe None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:01.948195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef1718-34b1-46bc-ad62-fb4487a430fe None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:01.948830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef1718-34b1-46bc-ad62-fb4487a430fe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:01.949132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef1718-34b1-46bc-ad62-fb4487a430fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:01.954336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef1718-34b1-46bc-ad62-fb4487a430fe None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:01.964135 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 288/595] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Oct 14 17:43:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:01.970191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef1718-34b1-46bc-ad62-fb4487a430fe None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:02.025427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:02.059508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f6ef8-6c8e-49f6-bd3d-19795681c50c None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:02.104367 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d3cb7-fafb-4103-9468-03368bc1b065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:02.105269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d3cb7-fafb-4103-9468-03368bc1b065 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:02.105544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d3cb7-fafb-4103-9468-03368bc1b065 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:02.105934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d3cb7-fafb-4103-9468-03368bc1b065 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:02.115783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d3cb7-fafb-4103-9468-03368bc1b065 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:02.118546 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d3cb7-fafb-4103-9468-03368bc1b065 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-212d3cb7-fafb-4103-9468-03368bc1b065 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:43:02.135829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:02.136934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:02.137961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 308/596] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Oct 14 17:43:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:02.139329 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d3cb7-fafb-4103-9468-03368bc1b065 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:02.141040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef1718-34b1-46bc-ad62-fb4487a430fe None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:02.186515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797bc3-2f13-4d64-a909-a12ee42612af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:02.187188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797bc3-2f13-4d64-a909-a12ee42612af None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:02.187566 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797bc3-2f13-4d64-a909-a12ee42612af None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:02.188092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797bc3-2f13-4d64-a909-a12ee42612af None admin] PATH_INFO: `/v3/endpoints` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:02.189690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57797bc3-2f13-4d64-a909-a12ee42612af None admin] RBAC: Adding query filter params (interface=public, service_id=26b3920f29274009b7ccfea92aecd437, region_id=RegionOne) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:02.190067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57797bc3-2f13-4d64-a909-a12ee42612af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:02.191502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57797bc3-2f13-4d64-a909-a12ee42612af None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:02.198654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 289/597] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Oct 14 17:43:01 2022] GET /identity/v3/endpoints?service_id=26b3920f29274009b7ccfea92aecd437&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:02.199978 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797bc3-2f13-4d64-a909-a12ee42612af None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:02.271144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:02.280061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d3cb7-fafb-4103-9468-03368bc1b065 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:02.314947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:02.322393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797bc3-2f13-4d64-a909-a12ee42612af None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:02.327371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f3715-6c17-4b78-ac1a-1df5981639b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:02.328025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f3715-6c17-4b78-ac1a-1df5981639b7 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:02.328302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f3715-6c17-4b78-ac1a-1df5981639b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:02.328572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f3715-6c17-4b78-ac1a-1df5981639b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:02.343067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f3715-6c17-4b78-ac1a-1df5981639b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-713f3715-6c17-4b78-ac1a-1df5981639b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:43:02.349413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:02.350539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:43:02.351393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:02.356157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 309/598] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:43:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:02.357005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f3715-6c17-4b78-ac1a-1df5981639b7 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:02.374493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4fa4f-d86b-4453-b0dc-6d1e8f5a7666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:02.375490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4fa4f-d86b-4453-b0dc-6d1e8f5a7666 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:02.376012 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4fa4f-d86b-4453-b0dc-6d1e8f5a7666 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:02.376510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4fa4f-d86b-4453-b0dc-6d1e8f5a7666 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:02.377337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4fa4f-d86b-4453-b0dc-6d1e8f5a7666 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:02.377787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4fa4f-d86b-4453-b0dc-6d1e8f5a7666 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:02.379304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4fa4f-d86b-4453-b0dc-6d1e8f5a7666 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:02.404885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 290/599] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Oct 14 17:43:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:02.405938 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4fa4f-d86b-4453-b0dc-6d1e8f5a7666 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:02.499053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f3715-6c17-4b78-ac1a-1df5981639b7 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:02.521440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4fa4f-d86b-4453-b0dc-6d1e8f5a7666 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:02.544692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:02.550891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af357535-ba7f-4115-961d-20eb7ea5f176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:02.551725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af357535-ba7f-4115-961d-20eb7ea5f176 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:02.552494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af357535-ba7f-4115-961d-20eb7ea5f176 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:02.552924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af357535-ba7f-4115-961d-20eb7ea5f176 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:02.554934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af357535-ba7f-4115-961d-20eb7ea5f176 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:02.557157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af357535-ba7f-4115-961d-20eb7ea5f176 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:02.564766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4644d-be9c-445a-af90-e97866e4dd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:02.564963 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 310/600] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Oct 14 17:43:02 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:02.565613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4644d-be9c-445a-af90-e97866e4dd9a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:02.565979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4644d-be9c-445a-af90-e97866e4dd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:02.566361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4644d-be9c-445a-af90-e97866e4dd9a None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:02.566733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af357535-ba7f-4115-961d-20eb7ea5f176 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:02.567248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4644d-be9c-445a-af90-e97866e4dd9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:02.567618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4644d-be9c-445a-af90-e97866e4dd9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:02.570088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4644d-be9c-445a-af90-e97866e4dd9a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:02.581056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 291/601] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:43:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:02.593106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4644d-be9c-445a-af90-e97866e4dd9a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:02.639694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:02.759044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af357535-ba7f-4115-961d-20eb7ea5f176 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:02.777087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4644d-be9c-445a-af90-e97866e4dd9a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:02.817148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b4fc-5b36-40b0-84b9-7886248b029a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:02.817780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b4fc-5b36-40b0-84b9-7886248b029a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:02.818053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b4fc-5b36-40b0-84b9-7886248b029a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:02.818324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b4fc-5b36-40b0-84b9-7886248b029a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:02.833208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed272ce8-cd8e-4eab-b32a-68113d529326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:02.833827 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed272ce8-cd8e-4eab-b32a-68113d529326 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:02.834105 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed272ce8-cd8e-4eab-b32a-68113d529326 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:02.834377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed272ce8-cd8e-4eab-b32a-68113d529326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:02.834897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b4fc-5b36-40b0-84b9-7886248b029a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:02.837608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b4fc-5b36-40b0-84b9-7886248b029a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-0e69b4fc-5b36-40b0-84b9-7886248b029a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:02.860071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:02.861112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:02.862021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:02.862021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:02.862021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:02.864306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 292/602] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:02.870596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b4fc-5b36-40b0-84b9-7886248b029a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:02.933544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:02.941408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed272ce8-cd8e-4eab-b32a-68113d529326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:02.943067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed272ce8-cd8e-4eab-b32a-68113d529326 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:02.995592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:03.049453 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b4fc-5b36-40b0-84b9-7886248b029a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:03.094635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 311/603] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:43:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:43:03.100236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed272ce8-cd8e-4eab-b32a-68113d529326 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:03.114013 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfbb3a-03bc-490e-94ee-fa4db7058d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:03.114624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfbb3a-03bc-490e-94ee-fa4db7058d0e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:03.114893 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfbb3a-03bc-490e-94ee-fa4db7058d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:03.115156 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfbb3a-03bc-490e-94ee-fa4db7058d0e None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:03.115793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfbb3a-03bc-490e-94ee-fa4db7058d0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:03.116683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfbb3a-03bc-490e-94ee-fa4db7058d0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:03.118006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfbb3a-03bc-490e-94ee-fa4db7058d0e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:03.139504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 293/604] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:03.146328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfbb3a-03bc-490e-94ee-fa4db7058d0e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:03.231974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed272ce8-cd8e-4eab-b32a-68113d529326 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:03.269162 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0299956-8555-492c-ac3a-83f1428b0b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:03.269802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0299956-8555-492c-ac3a-83f1428b0b1f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:03.270078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0299956-8555-492c-ac3a-83f1428b0b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:03.270358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0299956-8555-492c-ac3a-83f1428b0b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:03.280992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0299956-8555-492c-ac3a-83f1428b0b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:03.282445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0299956-8555-492c-ac3a-83f1428b0b1f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:03.295545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-a0299956-8555-492c-ac3a-83f1428b0b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:03.297779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:03.301345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:03.302288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:03.302288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:03.302288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:03.302288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 312/605] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:03.305004 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0299956-8555-492c-ac3a-83f1428b0b1f None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:03.307746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:03.312393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfbb3a-03bc-490e-94ee-fa4db7058d0e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:03.355923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44f235-e7f8-4764-8254-dfd8e1ac7a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:03.356839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44f235-e7f8-4764-8254-dfd8e1ac7a6b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:03.357327 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44f235-e7f8-4764-8254-dfd8e1ac7a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:03.357994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44f235-e7f8-4764-8254-dfd8e1ac7a6b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:03.369809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44f235-e7f8-4764-8254-dfd8e1ac7a6b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:03.371612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44f235-e7f8-4764-8254-dfd8e1ac7a6b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-cc44f235-e7f8-4764-8254-dfd8e1ac7a6b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:43:03.387502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:03.388789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:03.389889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:03.389889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:03.389889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:43:03.389889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:43:03.389889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:43:03.389889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:43:03.389889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:43:03.389889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:43:03.389889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 17:43:03.389889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:03.392342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 294/606] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:43:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:03.402594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44f235-e7f8-4764-8254-dfd8e1ac7a6b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:03.504167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0299956-8555-492c-ac3a-83f1428b0b1f None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:03.574573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f13f0-5f57-49ea-a0e8-cea5d18f3923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:03.575188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44f235-e7f8-4764-8254-dfd8e1ac7a6b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:03.575288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f13f0-5f57-49ea-a0e8-cea5d18f3923 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:03.575368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f13f0-5f57-49ea-a0e8-cea5d18f3923 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:03.575626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f13f0-5f57-49ea-a0e8-cea5d18f3923 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:03.576269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f13f0-5f57-49ea-a0e8-cea5d18f3923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:03.576836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f13f0-5f57-49ea-a0e8-cea5d18f3923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:03.579127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f13f0-5f57-49ea-a0e8-cea5d18f3923 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:03.596344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 313/607] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:03.597589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1dbb4-021f-4559-9ae6-8f48afa6dad8 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:03.597863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1dbb4-021f-4559-9ae6-8f48afa6dad8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:03.598667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1dbb4-021f-4559-9ae6-8f48afa6dad8 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:03.603988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 314/608] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:03.605107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1dbb4-021f-4559-9ae6-8f48afa6dad8 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:03.623495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1c1db-3dc5-4217-8cc7-df1173a65a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:03.624186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c1db-3dc5-4217-8cc7-df1173a65a0d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:03.624553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c1db-3dc5-4217-8cc7-df1173a65a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:03.624825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c1db-3dc5-4217-8cc7-df1173a65a0d None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:03.625705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1c1db-3dc5-4217-8cc7-df1173a65a0d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:03.626006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1c1db-3dc5-4217-8cc7-df1173a65a0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:03.627438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1c1db-3dc5-4217-8cc7-df1173a65a0d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:03.649454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 295/609] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Oct 14 17:43:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:03.650766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e87b-7b31-469d-8fb5-5b4333cb0560 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:03.651048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e87b-7b31-469d-8fb5-5b4333cb0560 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:03.651315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e87b-7b31-469d-8fb5-5b4333cb0560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:03.670341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-d999e87b-7b31-469d-8fb5-5b4333cb0560 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:03.671647 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d999e87b-7b31-469d-8fb5-5b4333cb0560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:03.673787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:03.680965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:03.725549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1dbb4-021f-4559-9ae6-8f48afa6dad8 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:03.761489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 296/610] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:03.762820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999e87b-7b31-469d-8fb5-5b4333cb0560 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:03.780086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a0299-ef95-4b49-ad00-fa2d9778f40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:03.780744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a0299-ef95-4b49-ad00-fa2d9778f40e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:03.781021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a0299-ef95-4b49-ad00-fa2d9778f40e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:03.781353 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a0299-ef95-4b49-ad00-fa2d9778f40e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:03.805228 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a0299-ef95-4b49-ad00-fa2d9778f40e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:03.806639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a0299-ef95-4b49-ad00-fa2d9778f40e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-171a0299-ef95-4b49-ad00-fa2d9778f40e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:43:03.826381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:03.827497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:03.828748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:43:03.828748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:43:03.828748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:43:03.828748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:43:03.828748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:43:03.828748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:43:03.828748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:03.828748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:03.835380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 315/611] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:03.837270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d9ab7-63af-408a-a9d8-51170f15c7cb None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:03.837622 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d9ab7-63af-408a-a9d8-51170f15c7cb None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:03.838015 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d9ab7-63af-408a-a9d8-51170f15c7cb None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:03.839528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 316/612] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:03.840248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d9ab7-63af-408a-a9d8-51170f15c7cb None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:03.887019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999e87b-7b31-469d-8fb5-5b4333cb0560 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:03.929039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b5f24-102c-4f7a-88da-cdd618b51050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:03.929939 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b5f24-102c-4f7a-88da-cdd618b51050 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:03.930398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b5f24-102c-4f7a-88da-cdd618b51050 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:03.930841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b5f24-102c-4f7a-88da-cdd618b51050 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:03.940237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:03.968481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d9ab7-63af-408a-a9d8-51170f15c7cb None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:03.998070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:04.011763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9afd1-55fe-431f-9ab8-ebd9275530df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:04.013559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9afd1-55fe-431f-9ab8-ebd9275530df None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:04.014277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9afd1-55fe-431f-9ab8-ebd9275530df None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:04.014722 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9afd1-55fe-431f-9ab8-ebd9275530df None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:04.014840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b5f24-102c-4f7a-88da-cdd618b51050 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:04.015705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9afd1-55fe-431f-9ab8-ebd9275530df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:04.016205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9afd1-55fe-431f-9ab8-ebd9275530df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:04.016309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b5f24-102c-4f7a-88da-cdd618b51050 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:04.018183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9afd1-55fe-431f-9ab8-ebd9275530df None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:04.034483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 317/613] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:43:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:04.036207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3580e9-7b83-442a-855d-3f5c7b89f8b6 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:04.037057 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3580e9-7b83-442a-855d-3f5c7b89f8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:04.037057 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3580e9-7b83-442a-855d-3f5c7b89f8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:04.060669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4a3580e9-7b83-442a-855d-3f5c7b89f8b6 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:04.062211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4a3580e9-7b83-442a-855d-3f5c7b89f8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:04.136321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 297/614] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:43:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:43:04.137244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b5f24-102c-4f7a-88da-cdd618b51050 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:04.143399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 318/615] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:04.144615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3580e9-7b83-442a-855d-3f5c7b89f8b6 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:04.183313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:04.259754 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b5f24-102c-4f7a-88da-cdd618b51050 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:04.259974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3580e9-7b83-442a-855d-3f5c7b89f8b6 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:04.275954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:04.301112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f936ba-42de-43ea-9497-c8b90dde68e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:04.302065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f936ba-42de-43ea-9497-c8b90dde68e1 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:04.302528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f936ba-42de-43ea-9497-c8b90dde68e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:04.303067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f936ba-42de-43ea-9497-c8b90dde68e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:04.303638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b9ab3-bad0-4fa7-a297-861c2b0ee71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:04.304256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f936ba-42de-43ea-9497-c8b90dde68e1 None admin] RBAC: Adding query filter params (role.id=e1a3bea83ac349acb6d65b6afb5441d6, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=d549c67a35a94c65a5cc2a3f3ec83474) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:04.304364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b9ab3-bad0-4fa7-a297-861c2b0ee71b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:04.304723 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b9ab3-bad0-4fa7-a297-861c2b0ee71b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:04.305040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f936ba-42de-43ea-9497-c8b90dde68e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:04.305142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b9ab3-bad0-4fa7-a297-861c2b0ee71b None admin] PATH_INFO: `/v3/services/network` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:04.305692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b9ab3-bad0-4fa7-a297-861c2b0ee71b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:04.306907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f936ba-42de-43ea-9497-c8b90dde68e1 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:04.307179 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b9ab3-bad0-4fa7-a297-861c2b0ee71b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:04.313742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 298/616] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:43:04 2022] GET /identity/v3/role_assignments?user.id=d549c67a35a94c65a5cc2a3f3ec83474&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=e1a3bea83ac349acb6d65b6afb5441d6 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:04.314799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f936ba-42de-43ea-9497-c8b90dde68e1 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-cc2b9ab3-bad0-4fa7-a297-861c2b0ee71b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:04.317295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:04.318352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 17:43:04.319771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:04.324487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 319/617] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:04 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:04.326630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b9ab3-bad0-4fa7-a297-861c2b0ee71b None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:04.490789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b9ab3-bad0-4fa7-a297-861c2b0ee71b None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:04.516763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f936ba-42de-43ea-9497-c8b90dde68e1 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:04.519348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:04.534100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe614d5-aef3-4d21-ac8b-23c2155dfb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:04.534711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe614d5-aef3-4d21-ac8b-23c2155dfb55 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:04.535954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe614d5-aef3-4d21-ac8b-23c2155dfb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:04.536235 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe614d5-aef3-4d21-ac8b-23c2155dfb55 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:04.536873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe614d5-aef3-4d21-ac8b-23c2155dfb55 None admin] RBAC: Adding query filter params (name=network) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:04.537174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe614d5-aef3-4d21-ac8b-23c2155dfb55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:04.538756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe614d5-aef3-4d21-ac8b-23c2155dfb55 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:04.544930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 320/618] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:43:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:04.545918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe614d5-aef3-4d21-ac8b-23c2155dfb55 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:04.562088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b28e0-cf49-4c3a-be04-be374a5c637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:04.563239 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b28e0-cf49-4c3a-be04-be374a5c637f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:04.563239 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b28e0-cf49-4c3a-be04-be374a5c637f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:04.563239 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b28e0-cf49-4c3a-be04-be374a5c637f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:04.579344 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b28e0-cf49-4c3a-be04-be374a5c637f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:04.580529 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b28e0-cf49-4c3a-be04-be374a5c637f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-bc6b28e0-cf49-4c3a-be04-be374a5c637f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:04.598432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:04.599452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:04.600593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:04.600593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:04.600593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:04.600593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 299/619] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:04.601290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b28e0-cf49-4c3a-be04-be374a5c637f None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:04.624731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:04.673001 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe614d5-aef3-4d21-ac8b-23c2155dfb55 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:04.707937 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b2264-462b-428d-90d5-269842b69e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:04.708698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b2264-462b-428d-90d5-269842b69e10 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:04.708982 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b2264-462b-428d-90d5-269842b69e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:04.709254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b2264-462b-428d-90d5-269842b69e10 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:04.710483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b2264-462b-428d-90d5-269842b69e10 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:04.711951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b2264-462b-428d-90d5-269842b69e10 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:04.717048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b28e0-cf49-4c3a-be04-be374a5c637f None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-c03b2264-462b-428d-90d5-269842b69e10 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:04.720841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:04.721794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:04.722691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 321/620] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Oct 14 17:43:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:04.723506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b2264-462b-428d-90d5-269842b69e10 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:04.759075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1348f-bc58-44d9-90cc-3a973864990b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:04.759811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1348f-bc58-44d9-90cc-3a973864990b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:04.760212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1348f-bc58-44d9-90cc-3a973864990b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:04.760578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1348f-bc58-44d9-90cc-3a973864990b None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:04.761354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1348f-bc58-44d9-90cc-3a973864990b None admin] RBAC: Adding query filter params (type=network) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:04.761744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1348f-bc58-44d9-90cc-3a973864990b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:04.763550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1348f-bc58-44d9-90cc-3a973864990b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:04.775075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 300/621] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:43:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:04.776173 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1348f-bc58-44d9-90cc-3a973864990b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:04.797291 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:04.837996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b2264-462b-428d-90d5-269842b69e10 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:04.840992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:04.876767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab43ddc-f52b-4bbe-a310-c5b4cf9c342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:04.877804 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab43ddc-f52b-4bbe-a310-c5b4cf9c342c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:04.878273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab43ddc-f52b-4bbe-a310-c5b4cf9c342c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:04.878718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab43ddc-f52b-4bbe-a310-c5b4cf9c342c None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:04.879674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab43ddc-f52b-4bbe-a310-c5b4cf9c342c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:04.880187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab43ddc-f52b-4bbe-a310-c5b4cf9c342c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:04.881996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab43ddc-f52b-4bbe-a310-c5b4cf9c342c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:04.886875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1348f-bc58-44d9-90cc-3a973864990b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:04.895681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 322/622] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:04.896981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab43ddc-f52b-4bbe-a310-c5b4cf9c342c None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:04.923536 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1477ed-f338-477b-a0e5-4f47908621d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:04.924114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1477ed-f338-477b-a0e5-4f47908621d5 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:04.924500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1477ed-f338-477b-a0e5-4f47908621d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:04.924840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1477ed-f338-477b-a0e5-4f47908621d5 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:04.925583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1477ed-f338-477b-a0e5-4f47908621d5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:04.925881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1477ed-f338-477b-a0e5-4f47908621d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:04.927241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1477ed-f338-477b-a0e5-4f47908621d5 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:04.935304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 301/623] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:04.935304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1477ed-f338-477b-a0e5-4f47908621d5 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:05.020641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab43ddc-f52b-4bbe-a310-c5b4cf9c342c None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:05.024639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:05.046554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1477ed-f338-477b-a0e5-4f47908621d5 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:05.068924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583312cc-40e0-441b-85b9-79903b20e34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:05.070696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583312cc-40e0-441b-85b9-79903b20e34d None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:05.071172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583312cc-40e0-441b-85b9-79903b20e34d None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:05.071848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583312cc-40e0-441b-85b9-79903b20e34d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:05.073279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583312cc-40e0-441b-85b9-79903b20e34d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:05.075166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583312cc-40e0-441b-85b9-79903b20e34d None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:05.083434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e488f2a-0f01-4f59-8031-c5da669cf8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:05.084081 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e488f2a-0f01-4f59-8031-c5da669cf8db None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:05.084358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e488f2a-0f01-4f59-8031-c5da669cf8db None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:05.084628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e488f2a-0f01-4f59-8031-c5da669cf8db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:05.092886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e488f2a-0f01-4f59-8031-c5da669cf8db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:05.094410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e488f2a-0f01-4f59-8031-c5da669cf8db None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-4e488f2a-0f01-4f59-8031-c5da669cf8db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:43:05.106746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:05.107762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:05.108735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:43:05.108735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:43:05.108735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:43:05.108735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:43:05.108735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:43:05.108735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:43:05.108735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:05.108735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:05.110320 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 323/624] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:43:04 2022] POST /identity/v3/endpoints => generated 337 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:05.111347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583312cc-40e0-441b-85b9-79903b20e34d None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:05.112271 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 302/625] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:05.117729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e488f2a-0f01-4f59-8031-c5da669cf8db None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:05.142331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:05.247072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583312cc-40e0-441b-85b9-79903b20e34d None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:05.274794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e488f2a-0f01-4f59-8031-c5da669cf8db None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:05.289438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cad75b-093b-4922-ace8-0e5c82fc9362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:05.290356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cad75b-093b-4922-ace8-0e5c82fc9362 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:05.290785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cad75b-093b-4922-ace8-0e5c82fc9362 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:05.291203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cad75b-093b-4922-ace8-0e5c82fc9362 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:05.292032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cad75b-093b-4922-ace8-0e5c82fc9362 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:05.292510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cad75b-093b-4922-ace8-0e5c82fc9362 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:05.294129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cad75b-093b-4922-ace8-0e5c82fc9362 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:05.299779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 324/626] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:05.305547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bd679-6a75-4aaf-ae42-f32df9d42a75 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:05.306055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bd679-6a75-4aaf-ae42-f32df9d42a75 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:05.306474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bd679-6a75-4aaf-ae42-f32df9d42a75 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:05.308278 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 325/627] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:05.314636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fdf03-a025-4062-b751-5dbeed94c0ec None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:05.315140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fdf03-a025-4062-b751-5dbeed94c0ec None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:05.315559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fdf03-a025-4062-b751-5dbeed94c0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:05.329221 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9541a6a-8ad3-494b-a916-509684dea77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:05.331911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9541a6a-8ad3-494b-a916-509684dea77d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:05.331911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9541a6a-8ad3-494b-a916-509684dea77d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:05.331911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9541a6a-8ad3-494b-a916-509684dea77d None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:05.331911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9541a6a-8ad3-494b-a916-509684dea77d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:05.331911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9541a6a-8ad3-494b-a916-509684dea77d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:05.332810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9541a6a-8ad3-494b-a916-509684dea77d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:05.339809 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-3e9fdf03-a025-4062-b751-5dbeed94c0ec None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:05.341107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3e9fdf03-a025-4062-b751-5dbeed94c0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:05.357835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 303/628] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:43:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:05.364265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9541a6a-8ad3-494b-a916-509684dea77d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:05.368697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:05.463820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 326/629] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:05.514286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9541a6a-8ad3-494b-a916-509684dea77d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:05.552617 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cb64a-85b1-4b05-9b49-229c14b325f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:05.553342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cb64a-85b1-4b05-9b49-229c14b325f8 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:05.553660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cb64a-85b1-4b05-9b49-229c14b325f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:05.554382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cb64a-85b1-4b05-9b49-229c14b325f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:05.554992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cb64a-85b1-4b05-9b49-229c14b325f8 None admin] RBAC: Adding query filter params (role.id=45b67ba9922c4b16baae3930d3de1b6c, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=9ec9c3fcf7964640a5eeabc01cd97617) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:05.555297 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cb64a-85b1-4b05-9b49-229c14b325f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:05.556759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cb64a-85b1-4b05-9b49-229c14b325f8 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:05.563179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 304/630] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:43:05 2022] GET /identity/v3/role_assignments?user.id=9ec9c3fcf7964640a5eeabc01cd97617&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=45b67ba9922c4b16baae3930d3de1b6c => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:05.758117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fdf03-a025-4062-b751-5dbeed94c0ec None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:05.770788 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:05.860272 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fdf03-a025-4062-b751-5dbeed94c0ec None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:05.895691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f1f96-242c-46fe-9f52-f6e1e52c401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:05.896351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f1f96-242c-46fe-9f52-f6e1e52c401e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:05.896681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f1f96-242c-46fe-9f52-f6e1e52c401e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:05.896953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f1f96-242c-46fe-9f52-f6e1e52c401e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:05.898169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f1f96-242c-46fe-9f52-f6e1e52c401e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:05.899549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f1f96-242c-46fe-9f52-f6e1e52c401e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-695f1f96-242c-46fe-9f52-f6e1e52c401e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:05.908316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:05.909482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:05.910541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 327/631] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Oct 14 17:43:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:05.915635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cb64a-85b1-4b05-9b49-229c14b325f8 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:05.983375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:06.021404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cb64a-85b1-4b05-9b49-229c14b325f8 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:06.056851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585a94a-2e62-4a73-838c-d7b2ebc6133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:06.057589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585a94a-2e62-4a73-838c-d7b2ebc6133d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:06.058024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585a94a-2e62-4a73-838c-d7b2ebc6133d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:06.058364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585a94a-2e62-4a73-838c-d7b2ebc6133d None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:06.059062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585a94a-2e62-4a73-838c-d7b2ebc6133d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:06.059506 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585a94a-2e62-4a73-838c-d7b2ebc6133d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:06.060975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585a94a-2e62-4a73-838c-d7b2ebc6133d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:06.066170 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 305/632] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Oct 14 17:43:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:06.072671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f1f96-242c-46fe-9f52-f6e1e52c401e None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:06.149740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:06.175268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f1f96-242c-46fe-9f52-f6e1e52c401e None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:06.210515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113210e-f872-4a89-a202-adaf7427f449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:06.211121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113210e-f872-4a89-a202-adaf7427f449 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:06.211392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113210e-f872-4a89-a202-adaf7427f449 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:06.211655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113210e-f872-4a89-a202-adaf7427f449 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:06.212313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113210e-f872-4a89-a202-adaf7427f449 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:06.212608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113210e-f872-4a89-a202-adaf7427f449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:06.214038 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113210e-f872-4a89-a202-adaf7427f449 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:06.219292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 328/633] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Oct 14 17:43:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:06.225031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585a94a-2e62-4a73-838c-d7b2ebc6133d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:06.330780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585a94a-2e62-4a73-838c-d7b2ebc6133d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:06.361822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:06.368258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0aa7f-bba3-43ea-b1a9-ba60907f4eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:06.369459 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0aa7f-bba3-43ea-b1a9-ba60907f4eba None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:06.369735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0aa7f-bba3-43ea-b1a9-ba60907f4eba None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:06.370006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0aa7f-bba3-43ea-b1a9-ba60907f4eba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:06.370699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0aa7f-bba3-43ea-b1a9-ba60907f4eba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:06.372208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0aa7f-bba3-43ea-b1a9-ba60907f4eba None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:06.402160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 306/634] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:43:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:06.696762 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69c2ed-f68f-43a8-a8ca-486f287e11ba None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:06.697054 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69c2ed-f68f-43a8-a8ca-486f287e11ba None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:06.697225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69c2ed-f68f-43a8-a8ca-486f287e11ba None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:06.698705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 329/635] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:06.704226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c16d-7c5b-4c8b-9658-ae79f7b48acf None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:06.704753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c16d-7c5b-4c8b-9658-ae79f7b48acf None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:06.705233 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c16d-7c5b-4c8b-9658-ae79f7b48acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:06.724385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1c17c16d-7c5b-4c8b-9658-ae79f7b48acf None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:06.725955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1c17c16d-7c5b-4c8b-9658-ae79f7b48acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:06.805722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 307/636] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:07.079762 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69c2ed-f68f-43a8-a8ca-486f287e11ba None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:07.186179 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69c2ed-f68f-43a8-a8ca-486f287e11ba None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:07.221658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecd415-d64a-4dd9-8bab-7133d66d11fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:07.222587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecd415-d64a-4dd9-8bab-7133d66d11fb None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:07.223053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecd415-d64a-4dd9-8bab-7133d66d11fb None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:07.223482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecd415-d64a-4dd9-8bab-7133d66d11fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:07.230935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:07.235219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecd415-d64a-4dd9-8bab-7133d66d11fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-b4ecd415-d64a-4dd9-8bab-7133d66d11fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:43:07.240544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:07.241598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 17:43:07.242486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:07.245881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 330/637] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Oct 14 17:43:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:07.248231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c16d-7c5b-4c8b-9658-ae79f7b48acf None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:07.355481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c16d-7c5b-4c8b-9658-ae79f7b48acf None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:07.369825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:07.392399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689136f-50b0-4a6d-a6ff-36b759bc0213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:07.393020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689136f-50b0-4a6d-a6ff-36b759bc0213 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:07.393302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689136f-50b0-4a6d-a6ff-36b759bc0213 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:07.393579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689136f-50b0-4a6d-a6ff-36b759bc0213 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:07.394253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689136f-50b0-4a6d-a6ff-36b759bc0213 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:07.399899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689136f-50b0-4a6d-a6ff-36b759bc0213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:07.399899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689136f-50b0-4a6d-a6ff-36b759bc0213 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:07.404762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 308/638] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Oct 14 17:43:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:07.410239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecd415-d64a-4dd9-8bab-7133d66d11fb None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:07.518955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecd415-d64a-4dd9-8bab-7133d66d11fb None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:07.557947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f600ee5-b20c-4028-80f7-6e2156bfa466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:07.558870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f600ee5-b20c-4028-80f7-6e2156bfa466 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:07.559345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f600ee5-b20c-4028-80f7-6e2156bfa466 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:07.559800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f600ee5-b20c-4028-80f7-6e2156bfa466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:07.571348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f600ee5-b20c-4028-80f7-6e2156bfa466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:07.573097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f600ee5-b20c-4028-80f7-6e2156bfa466 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-4f600ee5-b20c-4028-80f7-6e2156bfa466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:07.588989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:07.589974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:07.590880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:07.590880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:07.590880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:07.592827 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 331/639] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:07.604036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689136f-50b0-4a6d-a6ff-36b759bc0213 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:07.708817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689136f-50b0-4a6d-a6ff-36b759bc0213 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:07.743331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cada551-4ab9-429e-9e85-84d1ec1e0684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:07.743955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cada551-4ab9-429e-9e85-84d1ec1e0684 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:07.744229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cada551-4ab9-429e-9e85-84d1ec1e0684 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:07.744492 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cada551-4ab9-429e-9e85-84d1ec1e0684 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:07.745122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cada551-4ab9-429e-9e85-84d1ec1e0684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:07.745415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cada551-4ab9-429e-9e85-84d1ec1e0684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:07.746775 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cada551-4ab9-429e-9e85-84d1ec1e0684 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:07.764242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 309/640] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:07.766098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f600ee5-b20c-4028-80f7-6e2156bfa466 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:07.769111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5312-8805-4388-a642-a3593838fd50 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:07.769384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5312-8805-4388-a642-a3593838fd50 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:07.769646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5312-8805-4388-a642-a3593838fd50 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:07.771218 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 310/641] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:07.782321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98661e49-3762-4249-8daa-5647d0c0b7de None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:07.782653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98661e49-3762-4249-8daa-5647d0c0b7de None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:07.782920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98661e49-3762-4249-8daa-5647d0c0b7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:07.802660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-98661e49-3762-4249-8daa-5647d0c0b7de None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:07.803990 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-98661e49-3762-4249-8daa-5647d0c0b7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:07.826496 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:07.837646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:07.888902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 311/642] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:07.891411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f600ee5-b20c-4028-80f7-6e2156bfa466 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:07.926954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1ea3a-783a-46df-ad2e-b65ba17c6a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:07.927577 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1ea3a-783a-46df-ad2e-b65ba17c6a1e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:07.927850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1ea3a-783a-46df-ad2e-b65ba17c6a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:07.928144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1ea3a-783a-46df-ad2e-b65ba17c6a1e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:07.936006 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1ea3a-783a-46df-ad2e-b65ba17c6a1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:07.937464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1ea3a-783a-46df-ad2e-b65ba17c6a1e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:07.946007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7c392-7791-45bd-b3f1-1fbcc41ff1fe None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:07.946288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7c392-7791-45bd-b3f1-1fbcc41ff1fe None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:07.946558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7c392-7791-45bd-b3f1-1fbcc41ff1fe None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:07.948153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 312/643] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-81b1ea3a-783a-46df-ad2e-b65ba17c6a1e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:43:07.949428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:07.950388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:07.953992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 332/644] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Oct 14 17:43:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:07.957140 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666062-1719-4477-adc7-d279a0250d90 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:07.957424 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666062-1719-4477-adc7-d279a0250d90 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:07.957691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666062-1719-4477-adc7-d279a0250d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:07.958298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1ea3a-783a-46df-ad2e-b65ba17c6a1e None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:07.980459 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-78666062-1719-4477-adc7-d279a0250d90 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:07.982045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-78666062-1719-4477-adc7-d279a0250d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:08.070277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1ea3a-783a-46df-ad2e-b65ba17c6a1e None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:08.077152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 313/645] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:08.089689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:08.114420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8faf5-b5a4-4d74-a0be-c674ff9b4fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:08.115117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8faf5-b5a4-4d74-a0be-c674ff9b4fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:08.115394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8faf5-b5a4-4d74-a0be-c674ff9b4fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:08.115658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8faf5-b5a4-4d74-a0be-c674ff9b4fb3 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:08.116948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8faf5-b5a4-4d74-a0be-c674ff9b4fb3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:08.117246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8faf5-b5a4-4d74-a0be-c674ff9b4fb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:08.118668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8faf5-b5a4-4d74-a0be-c674ff9b4fb3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:08.137290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 333/646] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Oct 14 17:43:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:08.143486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78666062-1719-4477-adc7-d279a0250d90 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:08.162404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8faf5-b5a4-4d74-a0be-c674ff9b4fb3 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:08.245539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:08.253443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78666062-1719-4477-adc7-d279a0250d90 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:08.278706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8faf5-b5a4-4d74-a0be-c674ff9b4fb3 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:08.297970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ac02e-1f35-4006-824a-1f3f7adef989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:08.298596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ac02e-1f35-4006-824a-1f3f7adef989 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:08.298869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ac02e-1f35-4006-824a-1f3f7adef989 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:08.299132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ac02e-1f35-4006-824a-1f3f7adef989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:08.310869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ac02e-1f35-4006-824a-1f3f7adef989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:08.312452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ac02e-1f35-4006-824a-1f3f7adef989 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:08.323750 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2153-ba03-4d58-a0d5-6e57cb29670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:08.325032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2153-ba03-4d58-a0d5-6e57cb29670d None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:08.325723 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2153-ba03-4d58-a0d5-6e57cb29670d None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:08.326400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2153-ba03-4d58-a0d5-6e57cb29670d None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:08.327780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d2153-ba03-4d58-a0d5-6e57cb29670d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:08.329369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d2153-ba03-4d58-a0d5-6e57cb29670d None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-ee4ac02e-1f35-4006-824a-1f3f7adef989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:08.329510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:08.330403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:08.331313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:08.331313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:08.331313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:08.335280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 314/647] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:08.336790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ac02e-1f35-4006-824a-1f3f7adef989 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:08.355973 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 334/648] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:43:08 2022] POST /identity/v3/services => generated 269 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:43:08.389150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2153-ba03-4d58-a0d5-6e57cb29670d None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:08.424291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:08.497441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ac02e-1f35-4006-824a-1f3f7adef989 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:08.519525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2153-ba03-4d58-a0d5-6e57cb29670d None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:08.549191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a987e-ff72-4a90-a23d-4a8901bf6b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:08.549801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a987e-ff72-4a90-a23d-4a8901bf6b25 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:08.550073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a987e-ff72-4a90-a23d-4a8901bf6b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:08.550339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a987e-ff72-4a90-a23d-4a8901bf6b25 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:08.550969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a987e-ff72-4a90-a23d-4a8901bf6b25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:08.551262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a987e-ff72-4a90-a23d-4a8901bf6b25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:08.552865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a987e-ff72-4a90-a23d-4a8901bf6b25 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:08.561813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc6b52-c239-4074-a2f6-0feca1cf48e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:08.562750 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc6b52-c239-4074-a2f6-0feca1cf48e6 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:08.563238 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc6b52-c239-4074-a2f6-0feca1cf48e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:08.563712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc6b52-c239-4074-a2f6-0feca1cf48e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:08.564577 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc6b52-c239-4074-a2f6-0feca1cf48e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:08.566252 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc6b52-c239-4074-a2f6-0feca1cf48e6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:08.567299 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 315/649] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:08.572745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a987e-ff72-4a90-a23d-4a8901bf6b25 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-cefc6b52-c239-4074-a2f6-0feca1cf48e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:08.577575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:08.578603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:08.579473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:08.580914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 335/650] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:08.584066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:08.587913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc6b52-c239-4074-a2f6-0feca1cf48e6 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:08.700613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:08.706136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a987e-ff72-4a90-a23d-4a8901bf6b25 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:08.717489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc6b52-c239-4074-a2f6-0feca1cf48e6 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:08.744010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611f70a-0b1c-41c6-9d3f-427a98c56f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:08.744636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611f70a-0b1c-41c6-9d3f-427a98c56f73 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:08.744909 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611f70a-0b1c-41c6-9d3f-427a98c56f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:08.745162 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611f70a-0b1c-41c6-9d3f-427a98c56f73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:08.751973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611f70a-0b1c-41c6-9d3f-427a98c56f73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:08.753340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fca09-90e0-4ff7-93c2-199c4663373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:08.753478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611f70a-0b1c-41c6-9d3f-427a98c56f73 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:08.753940 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fca09-90e0-4ff7-93c2-199c4663373b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:08.754216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fca09-90e0-4ff7-93c2-199c4663373b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:08.754483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fca09-90e0-4ff7-93c2-199c4663373b None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:08.755269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fca09-90e0-4ff7-93c2-199c4663373b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:08.755566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fca09-90e0-4ff7-93c2-199c4663373b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:08.757832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fca09-90e0-4ff7-93c2-199c4663373b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:08.767268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 336/651] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:43:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:08.767268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b21c46-97c2-4f9d-9e86-841812ec8ffc None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:08.767268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b21c46-97c2-4f9d-9e86-841812ec8ffc None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:08.767268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b21c46-97c2-4f9d-9e86-841812ec8ffc None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-1611f70a-0b1c-41c6-9d3f-427a98c56f73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:43:08.767624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:08.768958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:08.769862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:43:08.769862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:43:08.769862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:43:08.769862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:43:08.769862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:43:08.769862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:43:08.769862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:08.769862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:08.769862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 316/652] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:08.769862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611f70a-0b1c-41c6-9d3f-427a98c56f73 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:08.770400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 337/653] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:08.771169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5eb0c-6c69-4944-99ce-d8b26d796465 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:08.771451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5eb0c-6c69-4944-99ce-d8b26d796465 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:08.771718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5eb0c-6c69-4944-99ce-d8b26d796465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:08.792968 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-6ee5eb0c-6c69-4944-99ce-d8b26d796465 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:08.794149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6ee5eb0c-6c69-4944-99ce-d8b26d796465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:08.841518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:08.865051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 338/654] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:08 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:08.866705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5eb0c-6c69-4944-99ce-d8b26d796465 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:08.894020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611f70a-0b1c-41c6-9d3f-427a98c56f73 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:08.935238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9bf0b-24d3-4def-98f3-ee7baae939f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:08.936025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9bf0b-24d3-4def-98f3-ee7baae939f8 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:08.936400 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9bf0b-24d3-4def-98f3-ee7baae939f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:08.936756 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9bf0b-24d3-4def-98f3-ee7baae939f8 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:08.937474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9bf0b-24d3-4def-98f3-ee7baae939f8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:08.937866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9bf0b-24d3-4def-98f3-ee7baae939f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:08.940178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9bf0b-24d3-4def-98f3-ee7baae939f8 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:08.947916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 317/655] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:43:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:08.977468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:08.999359 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5eb0c-6c69-4944-99ce-d8b26d796465 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:09.041214 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5d54a-41ba-491b-9bfb-9c5033e355f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:09.042108 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d54a-41ba-491b-9bfb-9c5033e355f1 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:09.042548 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d54a-41ba-491b-9bfb-9c5033e355f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:09.043005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d54a-41ba-491b-9bfb-9c5033e355f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:09.043857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5d54a-41ba-491b-9bfb-9c5033e355f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:09.044392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5d54a-41ba-491b-9bfb-9c5033e355f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:09.046183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5d54a-41ba-491b-9bfb-9c5033e355f1 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:09.064278 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 339/656] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:09.066517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9bf0b-24d3-4def-98f3-ee7baae939f8 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:09.155179 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5d54a-41ba-491b-9bfb-9c5033e355f1 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:09.186740 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9bf0b-24d3-4def-98f3-ee7baae939f8 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:09.218605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:09.225158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6a4b8-2094-407e-9cc1-d07fc04fcb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:09.226805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6a4b8-2094-407e-9cc1-d07fc04fcb5d None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:09.228106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6a4b8-2094-407e-9cc1-d07fc04fcb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:09.228591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6a4b8-2094-407e-9cc1-d07fc04fcb5d None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/d549c67a35a94c65a5cc2a3f3ec83474/roles/70d2eb11714847ce88bb184a459377d9` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:09.231535 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6a4b8-2094-407e-9cc1-d07fc04fcb5d None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=d549c67a35a94c65a5cc2a3f3ec83474, role_id=70d2eb11714847ce88bb184a459377d9)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:09.233069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6a4b8-2094-407e-9cc1-d07fc04fcb5d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:09.249782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 318/657] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:43:09 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/d549c67a35a94c65a5cc2a3f3ec83474/roles/70d2eb11714847ce88bb184a459377d9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:43:09.297220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5d54a-41ba-491b-9bfb-9c5033e355f1 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:09.337632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e1950-afda-4301-8050-0d4aa78f5c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:09.338294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e1950-afda-4301-8050-0d4aa78f5c03 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:09.338568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e1950-afda-4301-8050-0d4aa78f5c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:09.338888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e1950-afda-4301-8050-0d4aa78f5c03 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:09.339480 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e1950-afda-4301-8050-0d4aa78f5c03 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:09.340983 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e1950-afda-4301-8050-0d4aa78f5c03 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-5c5e1950-afda-4301-8050-0d4aa78f5c03 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:09.355913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:09.356998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:09.357856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 340/658] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:43:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:09.359806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6a4b8-2094-407e-9cc1-d07fc04fcb5d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:09.470815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6a4b8-2094-407e-9cc1-d07fc04fcb5d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:09.508133 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb684023-61cc-4b6e-ad95-fa9f020f0638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:09.508673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb684023-61cc-4b6e-ad95-fa9f020f0638 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:09.508950 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb684023-61cc-4b6e-ad95-fa9f020f0638 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:09.509222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb684023-61cc-4b6e-ad95-fa9f020f0638 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:09.509853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb684023-61cc-4b6e-ad95-fa9f020f0638 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:09.510154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb684023-61cc-4b6e-ad95-fa9f020f0638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:09.511515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb684023-61cc-4b6e-ad95-fa9f020f0638 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:09.516857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 319/659] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:43:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:09.522033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e1950-afda-4301-8050-0d4aa78f5c03 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:09.531998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:09.630305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e1950-afda-4301-8050-0d4aa78f5c03 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:09.666257 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69853cd-407b-44ef-9263-fc3e5ef90eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:09.666870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69853cd-407b-44ef-9263-fc3e5ef90eca None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:09.667146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69853cd-407b-44ef-9263-fc3e5ef90eca None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:09.667415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69853cd-407b-44ef-9263-fc3e5ef90eca None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:09.668173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69853cd-407b-44ef-9263-fc3e5ef90eca None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:09.668473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69853cd-407b-44ef-9263-fc3e5ef90eca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:09.669846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69853cd-407b-44ef-9263-fc3e5ef90eca None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:09.675123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 341/660] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:43:09 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:10.667071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a4f54-15bb-4ef9-84dd-d7317e73c49e None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:10.667604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a4f54-15bb-4ef9-84dd-d7317e73c49e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:10.667604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a4f54-15bb-4ef9-84dd-d7317e73c49e None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:10.669099 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 320/661] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:10.674518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3b61b-613d-4c8b-8065-3988e8fbd17e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:10.674925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3b61b-613d-4c8b-8065-3988e8fbd17e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:10.675105 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3b61b-613d-4c8b-8065-3988e8fbd17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:10.693670 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-89d3b61b-613d-4c8b-8065-3988e8fbd17e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:10.695579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-89d3b61b-613d-4c8b-8065-3988e8fbd17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:10.758703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:10.772036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 342/662] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:10 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:11.034078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a4f54-15bb-4ef9-84dd-d7317e73c49e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:11.092260 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:11.143084 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a4f54-15bb-4ef9-84dd-d7317e73c49e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:11.182107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccdc9b-c559-4bc4-8980-388a53213103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:11.182749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccdc9b-c559-4bc4-8980-388a53213103 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.183027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccdc9b-c559-4bc4-8980-388a53213103 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.183302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccdc9b-c559-4bc4-8980-388a53213103 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.183907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccdc9b-c559-4bc4-8980-388a53213103 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:11.185277 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccdc9b-c559-4bc4-8980-388a53213103 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-cbccdc9b-c559-4bc4-8980-388a53213103 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:11.194878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:11.195818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:11.196778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 321/663] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Oct 14 17:43:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:11.202009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3b61b-613d-4c8b-8065-3988e8fbd17e None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:11.283024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c795a7b-91bc-41ff-b908-37a99ac46a8c None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.283354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c795a7b-91bc-41ff-b908-37a99ac46a8c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.283864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c795a7b-91bc-41ff-b908-37a99ac46a8c None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.285649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 322/664] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:43:11.292478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f32f5e-9730-4602-9ab2-c46a678360ea None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.292753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f32f5e-9730-4602-9ab2-c46a678360ea None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.293025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f32f5e-9730-4602-9ab2-c46a678360ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.306306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3b61b-613d-4c8b-8065-3988e8fbd17e None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:11.315130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c3f32f5e-9730-4602-9ab2-c46a678360ea None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:11.316843 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c3f32f5e-9730-4602-9ab2-c46a678360ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:11.347333 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328352cd-b651-4325-9dbf-11eab9152410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:11.351061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328352cd-b651-4325-9dbf-11eab9152410 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.351061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328352cd-b651-4325-9dbf-11eab9152410 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.351061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328352cd-b651-4325-9dbf-11eab9152410 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.351061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328352cd-b651-4325-9dbf-11eab9152410 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:11.351061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328352cd-b651-4325-9dbf-11eab9152410 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:11.352415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328352cd-b651-4325-9dbf-11eab9152410 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:11.362545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 343/665] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:11.364098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328352cd-b651-4325-9dbf-11eab9152410 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:11.404995 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 323/666] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:11 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:11.448153 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:11.493117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328352cd-b651-4325-9dbf-11eab9152410 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:11.529729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b190614-259e-43b8-a0f9-008aa9b3557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:11.530671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b190614-259e-43b8-a0f9-008aa9b3557c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.531141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b190614-259e-43b8-a0f9-008aa9b3557c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.531624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b190614-259e-43b8-a0f9-008aa9b3557c None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.532498 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b190614-259e-43b8-a0f9-008aa9b3557c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:11.532993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b190614-259e-43b8-a0f9-008aa9b3557c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:11.534551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b190614-259e-43b8-a0f9-008aa9b3557c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:11.545279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 344/667] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:11.547478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f32f5e-9730-4602-9ab2-c46a678360ea None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:11.569779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d7225-c261-45ef-a860-251ee8f88aaf None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.570143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d7225-c261-45ef-a860-251ee8f88aaf None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.570346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d7225-c261-45ef-a860-251ee8f88aaf None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.571953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 345/668] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:11.576918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0633e6c-c750-422e-b5d2-9a0aeea492e7 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.577194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0633e6c-c750-422e-b5d2-9a0aeea492e7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.577461 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0633e6c-c750-422e-b5d2-9a0aeea492e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.597058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c0633e6c-c750-422e-b5d2-9a0aeea492e7 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:11.598253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c0633e6c-c750-422e-b5d2-9a0aeea492e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:11.606222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:11.670563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f32f5e-9730-4602-9ab2-c46a678360ea None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:11.691924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 346/669] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:11 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:11.697788 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0633e6c-c750-422e-b5d2-9a0aeea492e7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:11.716539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f44493-978c-41f3-a8d6-86ed40e1d6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:11.717429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f44493-978c-41f3-a8d6-86ed40e1d6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.717893 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f44493-978c-41f3-a8d6-86ed40e1d6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.718163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f44493-978c-41f3-a8d6-86ed40e1d6d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.718854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f44493-978c-41f3-a8d6-86ed40e1d6d0 None admin] RBAC: Adding query filter params (interface=public, service_id=d12b5f37b00845cca0b669fb85148580, region_id=RegionOne) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:11.719154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f44493-978c-41f3-a8d6-86ed40e1d6d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:11.720765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f44493-978c-41f3-a8d6-86ed40e1d6d0 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:11.731969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 324/670] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Oct 14 17:43:11 2022] GET /identity/v3/endpoints?service_id=d12b5f37b00845cca0b669fb85148580&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:11.734335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f44493-978c-41f3-a8d6-86ed40e1d6d0 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:11.768007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:11.819075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0633e6c-c750-422e-b5d2-9a0aeea492e7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:11.856588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f44493-978c-41f3-a8d6-86ed40e1d6d0 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:11.857488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b8c57-854c-49fe-b5e5-47b2c6859ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:11.858150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8c57-854c-49fe-b5e5-47b2c6859ac9 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.858424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8c57-854c-49fe-b5e5-47b2c6859ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.858689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8c57-854c-49fe-b5e5-47b2c6859ac9 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.859433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b8c57-854c-49fe-b5e5-47b2c6859ac9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:11.860918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b8c57-854c-49fe-b5e5-47b2c6859ac9 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:11.872400 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:11.887907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 347/671] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:43:11 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:43:11.914259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0d734-d091-4e07-b85f-2839fa8fca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:11.915004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0d734-d091-4e07-b85f-2839fa8fca33 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.915277 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0d734-d091-4e07-b85f-2839fa8fca33 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.915542 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0d734-d091-4e07-b85f-2839fa8fca33 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.916176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0d734-d091-4e07-b85f-2839fa8fca33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:11.917569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0d734-d091-4e07-b85f-2839fa8fca33 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:11.928508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 325/672] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Oct 14 17:43:11 2022] GET /identity/v3/services => generated 1258 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:43:11.930515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088e1cd-42ed-4650-b51b-4acda12399d8 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.931078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088e1cd-42ed-4650-b51b-4acda12399d8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.931586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088e1cd-42ed-4650-b51b-4acda12399d8 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.933618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 348/673] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:11.940007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c83131-4adf-4a5b-8f54-b5aaae8ef4f4 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:11.940340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c83131-4adf-4a5b-8f54-b5aaae8ef4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:11.940592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c83131-4adf-4a5b-8f54-b5aaae8ef4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:11.963039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-47c83131-4adf-4a5b-8f54-b5aaae8ef4f4 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:11.964647 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-47c83131-4adf-4a5b-8f54-b5aaae8ef4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:11.986531 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088e1cd-42ed-4650-b51b-4acda12399d8 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:12.065408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 326/674] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:11 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:12.114818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088e1cd-42ed-4650-b51b-4acda12399d8 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:12.133612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:12.154345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdec604-8124-404c-b835-ec8ddaaf74d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:12.155227 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdec604-8124-404c-b835-ec8ddaaf74d8 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:12.155655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdec604-8124-404c-b835-ec8ddaaf74d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:12.156115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdec604-8124-404c-b835-ec8ddaaf74d8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:12.167993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdec604-8124-404c-b835-ec8ddaaf74d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-dbdec604-8124-404c-b835-ec8ddaaf74d8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:43:12.173075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:12.174218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 17:43:12.175199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:12.176988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 349/675] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Oct 14 17:43:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:12.182788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c83131-4adf-4a5b-8f54-b5aaae8ef4f4 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:12.288821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c83131-4adf-4a5b-8f54-b5aaae8ef4f4 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:12.290432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:12.325561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34f609-cf87-435c-8265-ed7ca8a0727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:12.326235 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34f609-cf87-435c-8265-ed7ca8a0727d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:12.326510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34f609-cf87-435c-8265-ed7ca8a0727d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:12.326786 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34f609-cf87-435c-8265-ed7ca8a0727d None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:12.327501 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34f609-cf87-435c-8265-ed7ca8a0727d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:12.327802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34f609-cf87-435c-8265-ed7ca8a0727d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:12.329207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34f609-cf87-435c-8265-ed7ca8a0727d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:12.338452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 327/676] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Oct 14 17:43:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:12.342604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdec604-8124-404c-b835-ec8ddaaf74d8 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:12.344830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34f609-cf87-435c-8265-ed7ca8a0727d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:12.458473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34f609-cf87-435c-8265-ed7ca8a0727d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:12.476396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdec604-8124-404c-b835-ec8ddaaf74d8 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:12.499395 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18f1e2-1d1e-46ca-89c9-66b13bc90dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:12.500051 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18f1e2-1d1e-46ca-89c9-66b13bc90dce None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:12.500326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18f1e2-1d1e-46ca-89c9-66b13bc90dce None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:12.500597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18f1e2-1d1e-46ca-89c9-66b13bc90dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:12.512298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18f1e2-1d1e-46ca-89c9-66b13bc90dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:12.514055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18f1e2-1d1e-46ca-89c9-66b13bc90dce None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:12.523308 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c808cb-836b-40f8-b5ce-c42fe45df9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:12.524334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c808cb-836b-40f8-b5ce-c42fe45df9d2 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:12.524867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c808cb-836b-40f8-b5ce-c42fe45df9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:12.525287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c808cb-836b-40f8-b5ce-c42fe45df9d2 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:12.526124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c808cb-836b-40f8-b5ce-c42fe45df9d2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:12.527695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c808cb-836b-40f8-b5ce-c42fe45df9d2 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-3c18f1e2-1d1e-46ca-89c9-66b13bc90dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:12.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:12.531027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:12.532202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:12.532202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:12.532202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:12.537363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 328/677] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:12.539173 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18f1e2-1d1e-46ca-89c9-66b13bc90dce None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:12.552045 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 350/678] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:43:12 2022] POST /identity/v3/services => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:12.584288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:12.663785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18f1e2-1d1e-46ca-89c9-66b13bc90dce None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:12.708834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e23df-c6ca-4806-94bd-22de304f99c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:12.709481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e23df-c6ca-4806-94bd-22de304f99c7 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:12.709711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e23df-c6ca-4806-94bd-22de304f99c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:12.710430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e23df-c6ca-4806-94bd-22de304f99c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:12.711060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e23df-c6ca-4806-94bd-22de304f99c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:12.711384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e23df-c6ca-4806-94bd-22de304f99c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:12.712783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e23df-c6ca-4806-94bd-22de304f99c7 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:12.727021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 329/679] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:12.732727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c808cb-836b-40f8-b5ce-c42fe45df9d2 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:12.778016 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:12.845933 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c808cb-836b-40f8-b5ce-c42fe45df9d2 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:12.881717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98d6fd-64a3-4093-b9a3-ebf21d88f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:12.883313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98d6fd-64a3-4093-b9a3-ebf21d88f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:12.883797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98d6fd-64a3-4093-b9a3-ebf21d88f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:12.884302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98d6fd-64a3-4093-b9a3-ebf21d88f2c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:12.894019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98d6fd-64a3-4093-b9a3-ebf21d88f2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:12.895901 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98d6fd-64a3-4093-b9a3-ebf21d88f2c1 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-5e98d6fd-64a3-4093-b9a3-ebf21d88f2c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:43:12.909203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:12.910219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:12.911173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 351/680] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Oct 14 17:43:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:12.916889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e23df-c6ca-4806-94bd-22de304f99c7 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:12.992769 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:13.024013 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e23df-c6ca-4806-94bd-22de304f99c7 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:13.060701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7ce10-e8a7-42cd-b0c1-d184a8bd09b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:13.061304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ce10-e8a7-42cd-b0c1-d184a8bd09b6 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:13.061578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ce10-e8a7-42cd-b0c1-d184a8bd09b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:13.061848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ce10-e8a7-42cd-b0c1-d184a8bd09b6 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:13.062565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7ce10-e8a7-42cd-b0c1-d184a8bd09b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:13.062865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7ce10-e8a7-42cd-b0c1-d184a8bd09b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:13.064307 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7ce10-e8a7-42cd-b0c1-d184a8bd09b6 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:13.082805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 330/681] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Oct 14 17:43:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:13.088518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98d6fd-64a3-4093-b9a3-ebf21d88f2c1 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:13.192176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:13.199911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98d6fd-64a3-4093-b9a3-ebf21d88f2c1 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:13.237393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1db54e-1497-4433-9e9e-019b62d37472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:13.238372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1db54e-1497-4433-9e9e-019b62d37472 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:13.238858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1db54e-1497-4433-9e9e-019b62d37472 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:13.239419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1db54e-1497-4433-9e9e-019b62d37472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:13.313076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1db54e-1497-4433-9e9e-019b62d37472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:13.314877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1db54e-1497-4433-9e9e-019b62d37472 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:13.416896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:13.422386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 352/682] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:43:13 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:43:13.428128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7ce10-e8a7-42cd-b0c1-d184a8bd09b6 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:13.534194 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7ce10-e8a7-42cd-b0c1-d184a8bd09b6 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:13.564763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:13.571717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cac65-eefe-4883-b5f1-5f7de4637ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:13.572645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cac65-eefe-4883-b5f1-5f7de4637ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:13.573124 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cac65-eefe-4883-b5f1-5f7de4637ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:13.573615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cac65-eefe-4883-b5f1-5f7de4637ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:13.584585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cac65-eefe-4883-b5f1-5f7de4637ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:13.586966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cac65-eefe-4883-b5f1-5f7de4637ec6 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-9d9cac65-eefe-4883-b5f1-5f7de4637ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:13.603956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:13.605017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:13.605931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:13.605931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:13.605931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:13.608272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 331/683] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:13.614109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1db54e-1497-4433-9e9e-019b62d37472 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:13.719529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1db54e-1497-4433-9e9e-019b62d37472 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:13.756401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ef149-737d-41d9-8a5c-6ecc9cf4d90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:13.757915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ef149-737d-41d9-8a5c-6ecc9cf4d90a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:13.758419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ef149-737d-41d9-8a5c-6ecc9cf4d90a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:13.758889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ef149-737d-41d9-8a5c-6ecc9cf4d90a None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:13.759764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ef149-737d-41d9-8a5c-6ecc9cf4d90a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:13.760285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ef149-737d-41d9-8a5c-6ecc9cf4d90a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:13.761860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ef149-737d-41d9-8a5c-6ecc9cf4d90a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:13.776535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 353/684] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:13.782364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cac65-eefe-4883-b5f1-5f7de4637ec6 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:13.893594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cac65-eefe-4883-b5f1-5f7de4637ec6 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:13.929142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c90f64-1156-4116-9a98-0fa170be3589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:13.929780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c90f64-1156-4116-9a98-0fa170be3589 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:13.930112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c90f64-1156-4116-9a98-0fa170be3589 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:13.930385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c90f64-1156-4116-9a98-0fa170be3589 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:13.937591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c90f64-1156-4116-9a98-0fa170be3589 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:13.939044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c90f64-1156-4116-9a98-0fa170be3589 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-65c90f64-1156-4116-9a98-0fa170be3589 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:43:13.951361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:13.952338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:13.953254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:43:13.953254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:43:13.953254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:43:13.953254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:43:13.953254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:43:13.953254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:43:13.953254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:13.953254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:13.953254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 332/685] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:13.959687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ef149-737d-41d9-8a5c-6ecc9cf4d90a None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:14.024549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:14.070461 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ef149-737d-41d9-8a5c-6ecc9cf4d90a None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:14.117616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c663e7b-a086-4631-bbb4-b9425a806591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:14.118246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c663e7b-a086-4631-bbb4-b9425a806591 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:14.118524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c663e7b-a086-4631-bbb4-b9425a806591 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:14.118858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c663e7b-a086-4631-bbb4-b9425a806591 None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:14.119569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c663e7b-a086-4631-bbb4-b9425a806591 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:14.119941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c663e7b-a086-4631-bbb4-b9425a806591 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:14.121406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c663e7b-a086-4631-bbb4-b9425a806591 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:14.134697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 354/686] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:14.141004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c90f64-1156-4116-9a98-0fa170be3589 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:14.191219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:14.210956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719594a-44ee-460c-bdc8-05a9ca4476c8 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:14.211260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719594a-44ee-460c-bdc8-05a9ca4476c8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:14.211741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719594a-44ee-460c-bdc8-05a9ca4476c8 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:14.213250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 355/687] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:43:14.214099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736fb33-957e-485d-ab54-76f0fdc5e9fd None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:14.214369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736fb33-957e-485d-ab54-76f0fdc5e9fd None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:14.214633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736fb33-957e-485d-ab54-76f0fdc5e9fd None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:14.215956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 356/688] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:14.218130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31a3d2-2cf5-4362-a79c-25d16fe7a6d4 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:14.218605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31a3d2-2cf5-4362-a79c-25d16fe7a6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:14.219069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31a3d2-2cf5-4362-a79c-25d16fe7a6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:14.237657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ed31a3d2-2cf5-4362-a79c-25d16fe7a6d4 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:14.238842 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ed31a3d2-2cf5-4362-a79c-25d16fe7a6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:14.257631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c90f64-1156-4116-9a98-0fa170be3589 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:14.294100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11d7a-c5b6-4aeb-9ad7-afce4373e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:14.300746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11d7a-c5b6-4aeb-9ad7-afce4373e183 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:14.301090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11d7a-c5b6-4aeb-9ad7-afce4373e183 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:14.301850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11d7a-c5b6-4aeb-9ad7-afce4373e183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:14.302618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab11d7a-c5b6-4aeb-9ad7-afce4373e183 None admin] RBAC: Adding query filter params (role.id=70d2eb11714847ce88bb184a459377d9, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=d549c67a35a94c65a5cc2a3f3ec83474) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:14.302898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab11d7a-c5b6-4aeb-9ad7-afce4373e183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:14.304372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab11d7a-c5b6-4aeb-9ad7-afce4373e183 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:14.316200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 357/689] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:14.316200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ca45a-51cb-4f0a-9fee-8513fb495340 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:14.316200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ca45a-51cb-4f0a-9fee-8513fb495340 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:14.316583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 333/690] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:43:14 2022] GET /identity/v3/role_assignments?user.id=d549c67a35a94c65a5cc2a3f3ec83474&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=70d2eb11714847ce88bb184a459377d9 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:14.316672 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ca45a-51cb-4f0a-9fee-8513fb495340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:14.334653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0e2ca45a-51cb-4f0a-9fee-8513fb495340 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:14.337308 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0e2ca45a-51cb-4f0a-9fee-8513fb495340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:14.388606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:14.416076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 358/691] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:14.597211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11d7a-c5b6-4aeb-9ad7-afce4373e183 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:14.692137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ca45a-51cb-4f0a-9fee-8513fb495340 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:14.709231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11d7a-c5b6-4aeb-9ad7-afce4373e183 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:14.719221 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:14.761885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee20b9f-be6e-47bd-b976-bbab8879e8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:14.762916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee20b9f-be6e-47bd-b976-bbab8879e8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:14.763196 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee20b9f-be6e-47bd-b976-bbab8879e8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:14.763467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee20b9f-be6e-47bd-b976-bbab8879e8a0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:14.764604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee20b9f-be6e-47bd-b976-bbab8879e8a0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:14.765978 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee20b9f-be6e-47bd-b976-bbab8879e8a0 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-eee20b9f-be6e-47bd-b976-bbab8879e8a0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:14.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:14.779101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:14.780024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 334/692] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Oct 14 17:43:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:14.785425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee20b9f-be6e-47bd-b976-bbab8879e8a0 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:14.808892 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ca45a-51cb-4f0a-9fee-8513fb495340 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:14.844786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f9da5-ac7c-4768-b497-461b038c166a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:14.845410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f9da5-ac7c-4768-b497-461b038c166a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:14.845696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f9da5-ac7c-4768-b497-461b038c166a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:14.845939 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f9da5-ac7c-4768-b497-461b038c166a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:14.846513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f9da5-ac7c-4768-b497-461b038c166a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:14.847944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f9da5-ac7c-4768-b497-461b038c166a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-152f9da5-ac7c-4768-b497-461b038c166a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:14.856922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:14.857957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:14.858825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 359/693] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:14.865997 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f9da5-ac7c-4768-b497-461b038c166a None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:14.921122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee20b9f-be6e-47bd-b976-bbab8879e8a0 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:14.926925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:14.964899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa03e1-0e6b-417b-8071-a6cafd25e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:14.965851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa03e1-0e6b-417b-8071-a6cafd25e3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:14.966335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa03e1-0e6b-417b-8071-a6cafd25e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:14.966808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa03e1-0e6b-417b-8071-a6cafd25e3b6 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:14.967607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa03e1-0e6b-417b-8071-a6cafd25e3b6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:14.968142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa03e1-0e6b-417b-8071-a6cafd25e3b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:14.969818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa03e1-0e6b-417b-8071-a6cafd25e3b6 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:14.976750 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 335/694] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:14.982611 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa03e1-0e6b-417b-8071-a6cafd25e3b6 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:15.007453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f9da5-ac7c-4768-b497-461b038c166a None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:15.019783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:15.043797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f17346-5911-4f92-8a08-933081e353c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:15.044463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17346-5911-4f92-8a08-933081e353c6 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:15.044743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17346-5911-4f92-8a08-933081e353c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:15.045007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17346-5911-4f92-8a08-933081e353c6 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:15.045637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17346-5911-4f92-8a08-933081e353c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:15.046033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17346-5911-4f92-8a08-933081e353c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:15.047411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17346-5911-4f92-8a08-933081e353c6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:15.052287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 360/695] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:43:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:15.053254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc7936-5459-4ef9-af05-410cb1124a30 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:15.053536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc7936-5459-4ef9-af05-410cb1124a30 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:15.054327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc7936-5459-4ef9-af05-410cb1124a30 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:15.059631 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 361/696] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:15.060714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc7936-5459-4ef9-af05-410cb1124a30 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:15.104368 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa03e1-0e6b-417b-8071-a6cafd25e3b6 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:15.141715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465edec-872f-4e26-bf86-48b6dcec5ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:15.142316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465edec-872f-4e26-bf86-48b6dcec5ced None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:15.142587 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465edec-872f-4e26-bf86-48b6dcec5ced None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:15.142849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465edec-872f-4e26-bf86-48b6dcec5ced None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:15.143457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465edec-872f-4e26-bf86-48b6dcec5ced None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:15.144018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465edec-872f-4e26-bf86-48b6dcec5ced None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:15.145133 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465edec-872f-4e26-bf86-48b6dcec5ced None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:15.151199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 336/697] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:15.152302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f938ca4-0e99-419e-992a-da8ac6ee0edd None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:15.152577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f938ca4-0e99-419e-992a-da8ac6ee0edd None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:15.152845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f938ca4-0e99-419e-992a-da8ac6ee0edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:15.172245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6f938ca4-0e99-419e-992a-da8ac6ee0edd None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:15.173415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6f938ca4-0e99-419e-992a-da8ac6ee0edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:15.174516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc7936-5459-4ef9-af05-410cb1124a30 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:15.182066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:15.210888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301acace-0316-45df-bc22-7213117ea8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:15.211490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301acace-0316-45df-bc22-7213117ea8cd None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:15.211761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301acace-0316-45df-bc22-7213117ea8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:15.212064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301acace-0316-45df-bc22-7213117ea8cd None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:15.212697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301acace-0316-45df-bc22-7213117ea8cd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:15.212994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301acace-0316-45df-bc22-7213117ea8cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:15.214352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301acace-0316-45df-bc22-7213117ea8cd None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:15.219291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 362/698] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:43:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:15.220097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301acace-0316-45df-bc22-7213117ea8cd None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:15.253381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 337/699] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:15.254670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f938ca4-0e99-419e-992a-da8ac6ee0edd None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:15.302079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:15.341981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301acace-0316-45df-bc22-7213117ea8cd None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:15.382475 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4b941-4d14-4f6a-8515-1ddbecd47522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:15.383434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4b941-4d14-4f6a-8515-1ddbecd47522 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:15.384006 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4b941-4d14-4f6a-8515-1ddbecd47522 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:15.384474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4b941-4d14-4f6a-8515-1ddbecd47522 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:15.385329 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4b941-4d14-4f6a-8515-1ddbecd47522 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:15.387045 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4b941-4d14-4f6a-8515-1ddbecd47522 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:15.402546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f938ca4-0e99-419e-992a-da8ac6ee0edd None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:15.410899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 363/700] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:43:15 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:15.452744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de485091-75a2-40e9-8e8d-31b833a304c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:15.453736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de485091-75a2-40e9-8e8d-31b833a304c1 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:15.454362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de485091-75a2-40e9-8e8d-31b833a304c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:15.454852 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de485091-75a2-40e9-8e8d-31b833a304c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:15.455819 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de485091-75a2-40e9-8e8d-31b833a304c1 None admin] RBAC: Adding query filter params (interface=public, service_id=16cb4fb6206a4b5dbb9a91dee3bed32e, region_id=RegionOne) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:15.456431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de485091-75a2-40e9-8e8d-31b833a304c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:15.458102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de485091-75a2-40e9-8e8d-31b833a304c1 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:15.464898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 338/701] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Oct 14 17:43:15 2022] GET /identity/v3/endpoints?service_id=16cb4fb6206a4b5dbb9a91dee3bed32e&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:15.470598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4b941-4d14-4f6a-8515-1ddbecd47522 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:15.504087 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:15.546483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de485091-75a2-40e9-8e8d-31b833a304c1 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:15.592566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4b941-4d14-4f6a-8515-1ddbecd47522 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:15.630467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e324f93-f572-4395-82ec-3ff6e39d54a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:15.631080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324f93-f572-4395-82ec-3ff6e39d54a7 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:15.631357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324f93-f572-4395-82ec-3ff6e39d54a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:15.631675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324f93-f572-4395-82ec-3ff6e39d54a7 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:15.633147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e324f93-f572-4395-82ec-3ff6e39d54a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:15.634449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e324f93-f572-4395-82ec-3ff6e39d54a7 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:15.644890 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 364/702] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Oct 14 17:43:15 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:43:15.664550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:15.677398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de485091-75a2-40e9-8e8d-31b833a304c1 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:15.720436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb713304-3744-486d-953f-5dbffc44d4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:15.721519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb713304-3744-486d-953f-5dbffc44d4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:15.721997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb713304-3744-486d-953f-5dbffc44d4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:15.722525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb713304-3744-486d-953f-5dbffc44d4a1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:15.723336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb713304-3744-486d-953f-5dbffc44d4a1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:15.725207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb713304-3744-486d-953f-5dbffc44d4a1 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-eb713304-3744-486d-953f-5dbffc44d4a1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:15.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:15.736738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 17:43:15.737588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:15.740536 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 339/703] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:43:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:15.749301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e324f93-f572-4395-82ec-3ff6e39d54a7 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:15.849050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:15.861466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e324f93-f572-4395-82ec-3ff6e39d54a7 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:15.899257 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e964a-650e-4b7a-9c9f-1fa3b9f30257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:15.900154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e964a-650e-4b7a-9c9f-1fa3b9f30257 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:15.900581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e964a-650e-4b7a-9c9f-1fa3b9f30257 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:15.901023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e964a-650e-4b7a-9c9f-1fa3b9f30257 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:15.901811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e964a-650e-4b7a-9c9f-1fa3b9f30257 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:15.902309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e964a-650e-4b7a-9c9f-1fa3b9f30257 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:15.903821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e964a-650e-4b7a-9c9f-1fa3b9f30257 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:15.909286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 365/704] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:43:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:15.915977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb713304-3744-486d-953f-5dbffc44d4a1 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:16.024282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb713304-3744-486d-953f-5dbffc44d4a1 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:16.064809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404d405-cc25-4e34-9bf5-b9a431d91f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:16.065423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404d405-cc25-4e34-9bf5-b9a431d91f33 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:16.065699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404d405-cc25-4e34-9bf5-b9a431d91f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:16.065969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404d405-cc25-4e34-9bf5-b9a431d91f33 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:16.066600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404d405-cc25-4e34-9bf5-b9a431d91f33 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:16.066897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404d405-cc25-4e34-9bf5-b9a431d91f33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:16.068302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404d405-cc25-4e34-9bf5-b9a431d91f33 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:16.074399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 340/705] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:43:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:16.080343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e964a-650e-4b7a-9c9f-1fa3b9f30257 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:16.187055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e964a-650e-4b7a-9c9f-1fa3b9f30257 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:16.222509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bd6f9-5ca2-4d10-bc54-ae1a6e83442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:16.223536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd6f9-5ca2-4d10-bc54-ae1a6e83442d None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:16.224049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd6f9-5ca2-4d10-bc54-ae1a6e83442d None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:16.224578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd6f9-5ca2-4d10-bc54-ae1a6e83442d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:16.225504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bd6f9-5ca2-4d10-bc54-ae1a6e83442d None admin] RBAC: Adding query filter params (interface=public, service_id=e4ef5304c25e426b9267deff6fd3b913, region_id=RegionOne) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:16.226667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bd6f9-5ca2-4d10-bc54-ae1a6e83442d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:16.228375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bd6f9-5ca2-4d10-bc54-ae1a6e83442d None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:16.237067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 366/706] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Oct 14 17:43:16 2022] GET /identity/v3/endpoints?service_id=e4ef5304c25e426b9267deff6fd3b913&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:16.243669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404d405-cc25-4e34-9bf5-b9a431d91f33 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:16.264544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:16.353789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404d405-cc25-4e34-9bf5-b9a431d91f33 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:16.394543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606830dd-472e-44f9-a333-fd4f2abeb247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:16.395155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606830dd-472e-44f9-a333-fd4f2abeb247 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:16.395430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606830dd-472e-44f9-a333-fd4f2abeb247 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:16.395702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606830dd-472e-44f9-a333-fd4f2abeb247 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:16.396350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606830dd-472e-44f9-a333-fd4f2abeb247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:16.398293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606830dd-472e-44f9-a333-fd4f2abeb247 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:16.405207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 341/707] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Oct 14 17:43:16 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:43:16.628014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78d4fb-33c1-480d-8d66-c188651787e7 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:16.628345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78d4fb-33c1-480d-8d66-c188651787e7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:16.628681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78d4fb-33c1-480d-8d66-c188651787e7 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:16.630284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 367/708] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:16.636819 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e69dd-97f0-4ab7-9de2-4521a8af5263 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:16.637108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e69dd-97f0-4ab7-9de2-4521a8af5263 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:16.637363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e69dd-97f0-4ab7-9de2-4521a8af5263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:16.657928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f61e69dd-97f0-4ab7-9de2-4521a8af5263 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:16.659099 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f61e69dd-97f0-4ab7-9de2-4521a8af5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:16.731993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 342/709] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:17.006713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78d4fb-33c1-480d-8d66-c188651787e7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:17.028963 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:17.116869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78d4fb-33c1-480d-8d66-c188651787e7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:17.151754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281f897-175f-485a-baa1-93b0954f1297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:17.152411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281f897-175f-485a-baa1-93b0954f1297 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:17.152682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281f897-175f-485a-baa1-93b0954f1297 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:17.152949 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281f897-175f-485a-baa1-93b0954f1297 None admin] PATH_INFO: `/v3/services/image` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:17.153578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281f897-175f-485a-baa1-93b0954f1297 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:17.154950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281f897-175f-485a-baa1-93b0954f1297 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-a281f897-175f-485a-baa1-93b0954f1297 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:17.164439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:17.165472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 17:43:17.166334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:17.170542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 368/710] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Oct 14 17:43:17 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:17.171901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e69dd-97f0-4ab7-9de2-4521a8af5263 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:17.184290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:17.279366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e69dd-97f0-4ab7-9de2-4521a8af5263 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:17.314671 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ac7ed-9bc3-48f8-8c9d-a99867d75743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:17.315284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ac7ed-9bc3-48f8-8c9d-a99867d75743 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:17.315561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ac7ed-9bc3-48f8-8c9d-a99867d75743 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:17.315834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ac7ed-9bc3-48f8-8c9d-a99867d75743 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:17.316489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ac7ed-9bc3-48f8-8c9d-a99867d75743 None admin] RBAC: Adding query filter params (name=image) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:17.316790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ac7ed-9bc3-48f8-8c9d-a99867d75743 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:17.318153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ac7ed-9bc3-48f8-8c9d-a99867d75743 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:17.323363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 343/711] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:17.329041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281f897-175f-485a-baa1-93b0954f1297 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:17.424755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:17.445912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281f897-175f-485a-baa1-93b0954f1297 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:17.482194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659ee52-c2d7-4fa0-8eff-6e940b44c983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:17.483035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659ee52-c2d7-4fa0-8eff-6e940b44c983 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:17.483470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659ee52-c2d7-4fa0-8eff-6e940b44c983 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:17.483909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659ee52-c2d7-4fa0-8eff-6e940b44c983 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:17.484677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659ee52-c2d7-4fa0-8eff-6e940b44c983 None admin] RBAC: Adding query filter params (type=image) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:17.485138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659ee52-c2d7-4fa0-8eff-6e940b44c983 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:17.486639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659ee52-c2d7-4fa0-8eff-6e940b44c983 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:17.492495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 369/712] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:17 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:17.806961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cad27a-6d85-4b68-9158-54abd19dc179 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:17.807457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cad27a-6d85-4b68-9158-54abd19dc179 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:17.807842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cad27a-6d85-4b68-9158-54abd19dc179 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:17.809535 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 344/713] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:17.814966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76179ab2-b388-4764-9a1e-6bca014ba1ae None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:17.815377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76179ab2-b388-4764-9a1e-6bca014ba1ae None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:17.815749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76179ab2-b388-4764-9a1e-6bca014ba1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:17.834495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-76179ab2-b388-4764-9a1e-6bca014ba1ae None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:17.835764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-76179ab2-b388-4764-9a1e-6bca014ba1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:17.908798 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 370/714] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:18.024578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79426394-c783-4332-a393-7b6f02e4eb55 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:18.024900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79426394-c783-4332-a393-7b6f02e4eb55 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:18.025162 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79426394-c783-4332-a393-7b6f02e4eb55 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:18.026749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 345/715] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:18.032065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd89b1-7fd3-4b2a-be8e-c3b8855e5fda None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:18.032906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd89b1-7fd3-4b2a-be8e-c3b8855e5fda None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:18.033379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd89b1-7fd3-4b2a-be8e-c3b8855e5fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:18.054792 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-3dcd89b1-7fd3-4b2a-be8e-c3b8855e5fda None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:18.056379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3dcd89b1-7fd3-4b2a-be8e-c3b8855e5fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:18.126467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 371/716] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:18.192004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79426394-c783-4332-a393-7b6f02e4eb55 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:18.274532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:18.301461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79426394-c783-4332-a393-7b6f02e4eb55 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:18.340746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f5f54-3ff3-4142-8179-d5d0ead2467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:18.341758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f5f54-3ff3-4142-8179-d5d0ead2467b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:18.342236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f5f54-3ff3-4142-8179-d5d0ead2467b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:18.342702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f5f54-3ff3-4142-8179-d5d0ead2467b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:18.343548 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f5f54-3ff3-4142-8179-d5d0ead2467b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:18.345700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f5f54-3ff3-4142-8179-d5d0ead2467b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-5b3f5f54-3ff3-4142-8179-d5d0ead2467b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:18.358584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:18.359640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 17:43:18.360886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:18.363391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 346/717] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:18.369575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd89b1-7fd3-4b2a-be8e-c3b8855e5fda None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:18.380293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:18.420182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f5f54-3ff3-4142-8179-d5d0ead2467b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:18.480613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd89b1-7fd3-4b2a-be8e-c3b8855e5fda None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:18.516696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18c3b3-58a8-45a9-ae73-493761373c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:18.517329 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18c3b3-58a8-45a9-ae73-493761373c05 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:18.517660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18c3b3-58a8-45a9-ae73-493761373c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:18.517929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18c3b3-58a8-45a9-ae73-493761373c05 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:18.518604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18c3b3-58a8-45a9-ae73-493761373c05 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:18.518906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18c3b3-58a8-45a9-ae73-493761373c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:18.520372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18c3b3-58a8-45a9-ae73-493761373c05 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:18.525643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 372/718] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:43:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:18.531471 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18c3b3-58a8-45a9-ae73-493761373c05 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:18.539922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f5f54-3ff3-4142-8179-d5d0ead2467b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:18.581228 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c0355-f26c-46bb-9468-e602af86556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:18.581840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c0355-f26c-46bb-9468-e602af86556a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:18.582191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c0355-f26c-46bb-9468-e602af86556a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:18.582534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c0355-f26c-46bb-9468-e602af86556a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:18.583202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c0355-f26c-46bb-9468-e602af86556a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:18.584736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c0355-f26c-46bb-9468-e602af86556a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-ec1c0355-f26c-46bb-9468-e602af86556a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:18.596914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:18.598042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:18.598894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:18.604226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 347/719] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:18.606935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c0355-f26c-46bb-9468-e602af86556a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:18.609888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:18.648678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:18.657592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18c3b3-58a8-45a9-ae73-493761373c05 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:18.696073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec8266-700e-405a-b042-06ec0531f1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:18.697098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec8266-700e-405a-b042-06ec0531f1cd None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:18.697579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec8266-700e-405a-b042-06ec0531f1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:18.698057 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec8266-700e-405a-b042-06ec0531f1cd None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:18.698886 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec8266-700e-405a-b042-06ec0531f1cd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:18.699419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec8266-700e-405a-b042-06ec0531f1cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:18.701044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec8266-700e-405a-b042-06ec0531f1cd None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:18.706241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 373/720] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:43:18 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:18.716765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba67b1a-0cac-41c9-805a-4b72b7e9d117 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:18.717100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba67b1a-0cac-41c9-805a-4b72b7e9d117 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:18.717303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba67b1a-0cac-41c9-805a-4b72b7e9d117 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:18.723395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 374/721] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:18.724831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2fc73-7cf5-425f-885f-f2b04b56fe5c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:18.725111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2fc73-7cf5-425f-885f-f2b04b56fe5c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:18.725383 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2fc73-7cf5-425f-885f-f2b04b56fe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:18.735734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c0355-f26c-46bb-9468-e602af86556a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:18.748966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-3fd2fc73-7cf5-425f-885f-f2b04b56fe5c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:18.750406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3fd2fc73-7cf5-425f-885f-f2b04b56fe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:18.778096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b8b3b-dacf-474c-80ec-e92254d3978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:18.778723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b8b3b-dacf-474c-80ec-e92254d3978f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:18.779001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b8b3b-dacf-474c-80ec-e92254d3978f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:18.779273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b8b3b-dacf-474c-80ec-e92254d3978f None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:18.779944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b8b3b-dacf-474c-80ec-e92254d3978f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:18.780252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b8b3b-dacf-474c-80ec-e92254d3978f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:18.782285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b8b3b-dacf-474c-80ec-e92254d3978f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:18.793159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 348/722] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:43:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:18.794968 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b8b3b-dacf-474c-80ec-e92254d3978f None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:18.844833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 375/723] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:18.877002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:18.915457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b8b3b-dacf-474c-80ec-e92254d3978f None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:18.951526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59042672-2f06-4a67-82fe-990738b5a8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:18.952505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59042672-2f06-4a67-82fe-990738b5a8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:18.953000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59042672-2f06-4a67-82fe-990738b5a8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:18.953441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59042672-2f06-4a67-82fe-990738b5a8f5 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:18.954279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59042672-2f06-4a67-82fe-990738b5a8f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:18.954749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59042672-2f06-4a67-82fe-990738b5a8f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:18.956404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59042672-2f06-4a67-82fe-990738b5a8f5 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:18.965246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 349/724] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:43:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:18.968722 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2fc73-7cf5-425f-885f-f2b04b56fe5c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:19.037062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:19.078251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2fc73-7cf5-425f-885f-f2b04b56fe5c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:19.113101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b912b-c48d-4bf9-8675-5954ba46d1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:19.113708 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b912b-c48d-4bf9-8675-5954ba46d1c6 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:19.113980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b912b-c48d-4bf9-8675-5954ba46d1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:19.114245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b912b-c48d-4bf9-8675-5954ba46d1c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:19.114899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b912b-c48d-4bf9-8675-5954ba46d1c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:19.117033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b912b-c48d-4bf9-8675-5954ba46d1c6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:19.132080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59042672-2f06-4a67-82fe-990738b5a8f5 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:19.147915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 376/725] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:43:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:19.259001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59042672-2f06-4a67-82fe-990738b5a8f5 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:19.291355 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:19.305080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa63b74-5802-4d89-974c-0921e7a661f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:19.305727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa63b74-5802-4d89-974c-0921e7a661f5 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:19.306001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa63b74-5802-4d89-974c-0921e7a661f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:19.306266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa63b74-5802-4d89-974c-0921e7a661f5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:19.306847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa63b74-5802-4d89-974c-0921e7a661f5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:19.308268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa63b74-5802-4d89-974c-0921e7a661f5 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-4aa63b74-5802-4d89-974c-0921e7a661f5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:19.318089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:19.319152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:19.320725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 350/726] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:43:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:19.333035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b912b-c48d-4bf9-8675-5954ba46d1c6 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:19.457534 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b912b-c48d-4bf9-8675-5954ba46d1c6 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:19.477746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:19.494674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed746f-b176-4d12-8d61-76eabce67fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:19.495277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed746f-b176-4d12-8d61-76eabce67fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:19.495553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed746f-b176-4d12-8d61-76eabce67fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:19.495806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed746f-b176-4d12-8d61-76eabce67fb5 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:19.496461 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed746f-b176-4d12-8d61-76eabce67fb5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:19.496753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed746f-b176-4d12-8d61-76eabce67fb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:19.498129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed746f-b176-4d12-8d61-76eabce67fb5 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:19.508050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 377/727] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:43:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:19.510116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa63b74-5802-4d89-974c-0921e7a661f5 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:19.612240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa63b74-5802-4d89-974c-0921e7a661f5 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:19.651614 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4fa127-915f-4fb8-b456-4d1e2ea64f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:19.652269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fa127-915f-4fb8-b456-4d1e2ea64f4e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:19.652537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fa127-915f-4fb8-b456-4d1e2ea64f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:19.652799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fa127-915f-4fb8-b456-4d1e2ea64f4e None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:19.653505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4fa127-915f-4fb8-b456-4d1e2ea64f4e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:19.653803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4fa127-915f-4fb8-b456-4d1e2ea64f4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:19.655211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4fa127-915f-4fb8-b456-4d1e2ea64f4e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:19.666063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 351/728] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Oct 14 17:43:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:19.667515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed746f-b176-4d12-8d61-76eabce67fb5 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:19.780840 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed746f-b176-4d12-8d61-76eabce67fb5 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:19.781869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8eb2-3646-4e24-af93-5e14ad2d9be6 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:19.782152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8eb2-3646-4e24-af93-5e14ad2d9be6 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:19.782425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8eb2-3646-4e24-af93-5e14ad2d9be6 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:19.784155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 352/729] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:19.789508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9743c38-7384-4526-b6b8-8541090cdf3b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:19.790021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9743c38-7384-4526-b6b8-8541090cdf3b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:19.790511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9743c38-7384-4526-b6b8-8541090cdf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:19.809507 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f9743c38-7384-4526-b6b8-8541090cdf3b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:19.811258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f9743c38-7384-4526-b6b8-8541090cdf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:19.826209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95d204-2550-4169-b720-b500c15577b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:19.827152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95d204-2550-4169-b720-b500c15577b3 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:19.827482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95d204-2550-4169-b720-b500c15577b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:19.827751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95d204-2550-4169-b720-b500c15577b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:19.828925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95d204-2550-4169-b720-b500c15577b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:19.830586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95d204-2550-4169-b720-b500c15577b3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:19.859975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 378/730] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:43:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:19.886519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:19.905535 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 353/731] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:20.181892 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95d204-2550-4169-b720-b500c15577b3 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:20.227915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:20.296563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95d204-2550-4169-b720-b500c15577b3 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:20.337669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15d094-4ae7-4d99-9a9f-a23795b03d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:20.338656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d094-4ae7-4d99-9a9f-a23795b03d3c None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:20.339127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d094-4ae7-4d99-9a9f-a23795b03d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:20.339640 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d094-4ae7-4d99-9a9f-a23795b03d3c None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:20.340550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15d094-4ae7-4d99-9a9f-a23795b03d3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:20.342111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15d094-4ae7-4d99-9a9f-a23795b03d3c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:20.360556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 379/732] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:43:20 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:43:20.946899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba0da5-f846-482d-a590-b129d35de84b None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:20.947424 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba0da5-f846-482d-a590-b129d35de84b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:20.947424 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba0da5-f846-482d-a590-b129d35de84b None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:20.948998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 354/733] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:20.954386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b01b-69b2-40b6-9aad-26c6273ba61a None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:20.954671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b01b-69b2-40b6-9aad-26c6273ba61a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:20.954946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b01b-69b2-40b6-9aad-26c6273ba61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:20.973617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1046b01b-69b2-40b6-9aad-26c6273ba61a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:20.974789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1046b01b-69b2-40b6-9aad-26c6273ba61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:21.053175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 380/734] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:21.323121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba0da5-f846-482d-a590-b129d35de84b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:21.434247 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba0da5-f846-482d-a590-b129d35de84b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:21.472281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c3f7d-f51e-4d12-a062-cc3d6ea41d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:21.472876 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3f7d-f51e-4d12-a062-cc3d6ea41d8b None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:21.473144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3f7d-f51e-4d12-a062-cc3d6ea41d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:21.473409 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3f7d-f51e-4d12-a062-cc3d6ea41d8b None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:21.474056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c3f7d-f51e-4d12-a062-cc3d6ea41d8b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:21.475477 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c3f7d-f51e-4d12-a062-cc3d6ea41d8b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:21.494153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 355/735] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:43:21 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:21.545606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074c06b-629d-4bee-af33-39fdb07a214f None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:21.546170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074c06b-629d-4bee-af33-39fdb07a214f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:21.546170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074c06b-629d-4bee-af33-39fdb07a214f None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:21.547955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 381/736] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:21.554177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cd2c9-edeb-4568-bd9f-76e8a490e937 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:21.554611 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cd2c9-edeb-4568-bd9f-76e8a490e937 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:21.555075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cd2c9-edeb-4568-bd9f-76e8a490e937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:21.576363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f95cd2c9-edeb-4568-bd9f-76e8a490e937 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:21.577551 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f95cd2c9-edeb-4568-bd9f-76e8a490e937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:21.619256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:21.665765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 356/737] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:21 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:21.940797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074c06b-629d-4bee-af33-39fdb07a214f None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:21.970298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:22.056135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074c06b-629d-4bee-af33-39fdb07a214f None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:22.094827 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c89e3a-5a90-48a3-a39a-4c753f60f617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:22.095532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89e3a-5a90-48a3-a39a-4c753f60f617 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:22.095847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89e3a-5a90-48a3-a39a-4c753f60f617 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:22.096141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89e3a-5a90-48a3-a39a-4c753f60f617 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:22.096737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c89e3a-5a90-48a3-a39a-4c753f60f617 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:22.098141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c89e3a-5a90-48a3-a39a-4c753f60f617 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-79c89e3a-5a90-48a3-a39a-4c753f60f617 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:22.107777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:22.108878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:22.109748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 382/738] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:22.115052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cd2c9-edeb-4568-bd9f-76e8a490e937 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:22.210268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d077-a254-456b-bcc4-f97b9fd8d7f5 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:22.210936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d077-a254-456b-bcc4-f97b9fd8d7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:22.211418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d077-a254-456b-bcc4-f97b9fd8d7f5 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:22.213348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 383/739] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:43:22.218998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70c885-5bf9-4436-8982-ebd8a87ba322 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:22.219548 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70c885-5bf9-4436-8982-ebd8a87ba322 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:22.220159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70c885-5bf9-4436-8982-ebd8a87ba322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:22.224725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cd2c9-edeb-4568-bd9f-76e8a490e937 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:22.240210 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8d70c885-5bf9-4436-8982-ebd8a87ba322 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:22.242608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8d70c885-5bf9-4436-8982-ebd8a87ba322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:22.264919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c49df4-429c-43f0-8fd4-80384eb846e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:22.265530 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c49df4-429c-43f0-8fd4-80384eb846e9 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:22.265806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c49df4-429c-43f0-8fd4-80384eb846e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:22.266077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c49df4-429c-43f0-8fd4-80384eb846e9 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:22.266701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c49df4-429c-43f0-8fd4-80384eb846e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:22.266997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c49df4-429c-43f0-8fd4-80384eb846e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:22.268392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c49df4-429c-43f0-8fd4-80384eb846e9 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:22.274920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 357/740] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:43:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:22.281307 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c49df4-429c-43f0-8fd4-80384eb846e9 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:22.315278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:22.342950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 384/741] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:22 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:22.414711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c49df4-429c-43f0-8fd4-80384eb846e9 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:22.451331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776efdbf-019e-4be4-9f7f-ae332509cd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:22.452374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776efdbf-019e-4be4-9f7f-ae332509cd63 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:22.452861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776efdbf-019e-4be4-9f7f-ae332509cd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:22.453332 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776efdbf-019e-4be4-9f7f-ae332509cd63 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:22.454240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776efdbf-019e-4be4-9f7f-ae332509cd63 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:22.454792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776efdbf-019e-4be4-9f7f-ae332509cd63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:22.456424 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776efdbf-019e-4be4-9f7f-ae332509cd63 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:22.463521 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 358/742] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:43:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:22.469202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70c885-5bf9-4436-8982-ebd8a87ba322 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:22.495177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:22.581308 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70c885-5bf9-4436-8982-ebd8a87ba322 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:22.599596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079784be-2678-4d4e-bf78-73a7468deef0 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:22.599919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079784be-2678-4d4e-bf78-73a7468deef0 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:22.600177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079784be-2678-4d4e-bf78-73a7468deef0 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:22.602580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 359/743] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:43:22.607494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ad3dc-30f9-4592-838c-04d7db98ab43 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:22.607777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ad3dc-30f9-4592-838c-04d7db98ab43 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:22.608088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ad3dc-30f9-4592-838c-04d7db98ab43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:22.625475 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224986c-cce0-410d-b4a4-7b3f0df22b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:22.626398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224986c-cce0-410d-b4a4-7b3f0df22b27 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:22.626779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-8b6ad3dc-30f9-4592-838c-04d7db98ab43 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:22.627043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224986c-cce0-410d-b4a4-7b3f0df22b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:22.627455 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224986c-cce0-410d-b4a4-7b3f0df22b27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:22.627991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8b6ad3dc-30f9-4592-838c-04d7db98ab43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:22.628503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224986c-cce0-410d-b4a4-7b3f0df22b27 None admin] RBAC: Adding query filter params (interface=admin, service_id=16cb4fb6206a4b5dbb9a91dee3bed32e, region_id=RegionOne) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:22.628973 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224986c-cce0-410d-b4a4-7b3f0df22b27 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:22.631626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224986c-cce0-410d-b4a4-7b3f0df22b27 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:22.638561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 385/744] 10.222.0.27 () {56 vars in 1249 bytes} [Fri Oct 14 17:43:22 2022] GET /identity/v3/endpoints?service_id=16cb4fb6206a4b5dbb9a91dee3bed32e&interface=admin®ion_id=RegionOne => generated 193 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:22.642312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224986c-cce0-410d-b4a4-7b3f0df22b27 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:22.700086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 360/745] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:22 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:22.701395 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ad3dc-30f9-4592-838c-04d7db98ab43 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:22.729549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:22.740597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:22.759924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224986c-cce0-410d-b4a4-7b3f0df22b27 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:22.797759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:22.798621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:22.799041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:22.799404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:22.800228 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:22.801754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:22.805280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WARNING keystone.server.flask.common [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 14 17:43:22.815602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ad3dc-30f9-4592-838c-04d7db98ab43 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:22.825358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.sql.core [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 14 17:43:22.825358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 17:43:22.825358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [parameters: {'id': '9212c2ef8e2b480c9c4a27d1d1a7ca35', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 14 17:43:22.825358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81267) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9212c2ef8e2b480c9c4a27d1d1a7ca35.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9212c2ef8e2b480c9c4a27d1d1a7ca35. Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application conn.query(q) Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 17:43:22.834068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result.read() Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 14 17:43:22.835095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application next(self.gen) Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application yield resource Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application next(self.gen) Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise self.value Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application session.commit() Oct 14 17:43:22.836282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.session.flush() Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application compat.raise_( Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 17:43:22.837411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 17:43:22.839186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application conn.query(q) Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result.read() Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 17:43:22.841261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [parameters: {'id': '9212c2ef8e2b480c9c4a27d1d1a7ca35', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:22.843776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:22.845960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9212c2ef8e2b480c9c4a27d1d1a7ca35. Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 386/746] 10.222.0.27 () {60 vars in 1140 bytes} [Fri Oct 14 17:43:22 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:22.848192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:22.869358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d1b43-0b28-4a38-adb1-72338dbf90e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:22.870434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d1b43-0b28-4a38-adb1-72338dbf90e4 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:22.871217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d1b43-0b28-4a38-adb1-72338dbf90e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:22.871689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d1b43-0b28-4a38-adb1-72338dbf90e4 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:22.872868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d1b43-0b28-4a38-adb1-72338dbf90e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:22.874337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d1b43-0b28-4a38-adb1-72338dbf90e4 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:22.882466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 361/747] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Oct 14 17:43:22 2022] GET /identity/v3/services => generated 1957 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:43:22.977373 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815f95f-c376-4460-866c-fa8ee55d703b None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:22.979639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d1b43-0b28-4a38-adb1-72338dbf90e4 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:23.018683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20378bb5-bb77-45ea-9525-34d8db1f85d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:23.019318 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20378bb5-bb77-45ea-9525-34d8db1f85d0 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:23.019594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20378bb5-bb77-45ea-9525-34d8db1f85d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:23.019852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20378bb5-bb77-45ea-9525-34d8db1f85d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:23.028021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20378bb5-bb77-45ea-9525-34d8db1f85d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:23.030186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20378bb5-bb77-45ea-9525-34d8db1f85d0 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:23.033838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-20378bb5-bb77-45ea-9525-34d8db1f85d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:43:23.045470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:23.046442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:23.047395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:43:23.047395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:43:23.047395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:43:23.047395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:43:23.047395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:43:23.047395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:43:23.047395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:23.047395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:23.051382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 387/748] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:23.052926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20378bb5-bb77-45ea-9525-34d8db1f85d0 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:23.111942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:23.128551 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d1b43-0b28-4a38-adb1-72338dbf90e4 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:23.169766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a3e6e-2b8f-4e22-a442-cfe6bd387d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:23.170429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a3e6e-2b8f-4e22-a442-cfe6bd387d45 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:23.170705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a3e6e-2b8f-4e22-a442-cfe6bd387d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:23.170975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a3e6e-2b8f-4e22-a442-cfe6bd387d45 None admin] PATH_INFO: `/v3/services/image` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:23.171806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a3e6e-2b8f-4e22-a442-cfe6bd387d45 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:23.173986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a3e6e-2b8f-4e22-a442-cfe6bd387d45 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:23.183167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20378bb5-bb77-45ea-9525-34d8db1f85d0 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-548a3e6e-2b8f-4e22-a442-cfe6bd387d45 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:23.185010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:23.186017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:23.186886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 362/749] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Oct 14 17:43:22 2022] GET /identity/v3/services/image => generated 86 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:23.191977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a3e6e-2b8f-4e22-a442-cfe6bd387d45 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:23.229927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b568850-ffa3-4eab-b4ba-4a9328da0f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:23.231112 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568850-ffa3-4eab-b4ba-4a9328da0f5e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:23.231598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568850-ffa3-4eab-b4ba-4a9328da0f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:23.232874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568850-ffa3-4eab-b4ba-4a9328da0f5e None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:23.233780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568850-ffa3-4eab-b4ba-4a9328da0f5e None admin] RBAC: Adding query filter params (name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:23.234288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568850-ffa3-4eab-b4ba-4a9328da0f5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:23.235979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568850-ffa3-4eab-b4ba-4a9328da0f5e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:23.264079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 388/750] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:43:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:23.312480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a3e6e-2b8f-4e22-a442-cfe6bd387d45 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:23.354408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480781-72ea-4b12-a3d9-45256720d909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:23.355035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480781-72ea-4b12-a3d9-45256720d909 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:23.355313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480781-72ea-4b12-a3d9-45256720d909 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:23.355585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480781-72ea-4b12-a3d9-45256720d909 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:23.356259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480781-72ea-4b12-a3d9-45256720d909 None admin] RBAC: Adding query filter params (name=image) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:23.356561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480781-72ea-4b12-a3d9-45256720d909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:23.357958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480781-72ea-4b12-a3d9-45256720d909 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:23.364277 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 363/751] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:23.369916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b568850-ffa3-4eab-b4ba-4a9328da0f5e None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:23.441411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:23.486171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b568850-ffa3-4eab-b4ba-4a9328da0f5e None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:23.520991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49763796-3e47-44a7-bd27-d14c9c8b6574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:23.521596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49763796-3e47-44a7-bd27-d14c9c8b6574 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:23.521867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49763796-3e47-44a7-bd27-d14c9c8b6574 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:23.522129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49763796-3e47-44a7-bd27-d14c9c8b6574 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:23.522781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49763796-3e47-44a7-bd27-d14c9c8b6574 None admin] RBAC: Adding query filter params (type=image) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:23.523305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49763796-3e47-44a7-bd27-d14c9c8b6574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:23.524772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49763796-3e47-44a7-bd27-d14c9c8b6574 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:23.531448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 389/752] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:23.536216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480781-72ea-4b12-a3d9-45256720d909 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:23.558318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:23.646106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480781-72ea-4b12-a3d9-45256720d909 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:23.681861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cda441-264e-4e6a-ab45-b478135e3a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:23.682840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cda441-264e-4e6a-ab45-b478135e3a95 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:23.683308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cda441-264e-4e6a-ab45-b478135e3a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:23.683786 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cda441-264e-4e6a-ab45-b478135e3a95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:23.684721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cda441-264e-4e6a-ab45-b478135e3a95 None admin] RBAC: Adding query filter params (interface=public, service_id=78d69c7854b94f3287356d604416cd08, region_id=RegionOne) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:23.685220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cda441-264e-4e6a-ab45-b478135e3a95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:23.686805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cda441-264e-4e6a-ab45-b478135e3a95 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:23.697856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 364/753] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Oct 14 17:43:23 2022] GET /identity/v3/endpoints?service_id=78d69c7854b94f3287356d604416cd08&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:23.701100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49763796-3e47-44a7-bd27-d14c9c8b6574 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:23.809638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49763796-3e47-44a7-bd27-d14c9c8b6574 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:23.823622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fc225-255e-4088-a796-8a93f803008a None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:23.823938 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fc225-255e-4088-a796-8a93f803008a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:23.824208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fc225-255e-4088-a796-8a93f803008a None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:23.825765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 365/754] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:43:23.830755 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142cbd7-245c-41c9-8961-1996fc04bdf9 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:23.831033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142cbd7-245c-41c9-8961-1996fc04bdf9 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:23.831302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142cbd7-245c-41c9-8961-1996fc04bdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:23.834325 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:23.849164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48050b9f-37a9-4f75-8529-1eb85fb9a693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:23.850305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48050b9f-37a9-4f75-8529-1eb85fb9a693 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:23.850872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48050b9f-37a9-4f75-8529-1eb85fb9a693 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:23.851288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48050b9f-37a9-4f75-8529-1eb85fb9a693 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:23.852090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48050b9f-37a9-4f75-8529-1eb85fb9a693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:23.853646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48050b9f-37a9-4f75-8529-1eb85fb9a693 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:23.854078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6142cbd7-245c-41c9-8961-1996fc04bdf9 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:23.855613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6142cbd7-245c-41c9-8961-1996fc04bdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:23.863115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 390/755] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Oct 14 17:43:23 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:43:23.942130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 366/756] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:23 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:24.215194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48050b9f-37a9-4f75-8529-1eb85fb9a693 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:24.328989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48050b9f-37a9-4f75-8529-1eb85fb9a693 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:24.364797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9b4e3-d87b-4307-b274-5fa8829af4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:24.365432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9b4e3-d87b-4307-b274-5fa8829af4cc None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:24.365707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9b4e3-d87b-4307-b274-5fa8829af4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:24.366072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9b4e3-d87b-4307-b274-5fa8829af4cc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:24.367598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9b4e3-d87b-4307-b274-5fa8829af4cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:24.368790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9b4e3-d87b-4307-b274-5fa8829af4cc None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-9ea9b4e3-d87b-4307-b274-5fa8829af4cc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:24.377665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:24.378649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:24.379504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 391/757] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:24.392916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142cbd7-245c-41c9-8961-1996fc04bdf9 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:24.405890 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:24.497629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142cbd7-245c-41c9-8961-1996fc04bdf9 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:24.532117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934289c1-a932-4a8c-9f67-b0b4b5a15624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:24.533067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934289c1-a932-4a8c-9f67-b0b4b5a15624 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:24.533538 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934289c1-a932-4a8c-9f67-b0b4b5a15624 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:24.533999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934289c1-a932-4a8c-9f67-b0b4b5a15624 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:24.534823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934289c1-a932-4a8c-9f67-b0b4b5a15624 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:24.535381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934289c1-a932-4a8c-9f67-b0b4b5a15624 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:24.536953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934289c1-a932-4a8c-9f67-b0b4b5a15624 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:24.546773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 367/758] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:43:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:24.552198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9b4e3-d87b-4307-b274-5fa8829af4cc None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:24.614197 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:24.664609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9b4e3-d87b-4307-b274-5fa8829af4cc None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:24.699706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241bf21-5de3-4281-953b-8e3cbf08c49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:24.700339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241bf21-5de3-4281-953b-8e3cbf08c49c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:24.700617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241bf21-5de3-4281-953b-8e3cbf08c49c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:24.700885 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241bf21-5de3-4281-953b-8e3cbf08c49c None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:24.701511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241bf21-5de3-4281-953b-8e3cbf08c49c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:24.702030 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241bf21-5de3-4281-953b-8e3cbf08c49c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:24.703167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241bf21-5de3-4281-953b-8e3cbf08c49c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:24.708252 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 392/759] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:43:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:24.715654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934289c1-a932-4a8c-9f67-b0b4b5a15624 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:24.798207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:24.826476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934289c1-a932-4a8c-9f67-b0b4b5a15624 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:24.861443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db31a9-1790-45e2-a28f-305ee9134571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:24.862042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db31a9-1790-45e2-a28f-305ee9134571 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:24.862314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db31a9-1790-45e2-a28f-305ee9134571 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:24.862582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db31a9-1790-45e2-a28f-305ee9134571 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:24.863274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db31a9-1790-45e2-a28f-305ee9134571 None admin] RBAC: Adding query filter params (interface=public, service_id=c8a443a2c38c4fafaa3802bfb4d06e7a, region_id=RegionOne) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:24.863575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db31a9-1790-45e2-a28f-305ee9134571 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:24.864965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db31a9-1790-45e2-a28f-305ee9134571 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:24.870343 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 368/760] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Oct 14 17:43:24 2022] GET /identity/v3/endpoints?service_id=c8a443a2c38c4fafaa3802bfb4d06e7a&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:24.875600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241bf21-5de3-4281-953b-8e3cbf08c49c None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:24.983101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241bf21-5de3-4281-953b-8e3cbf08c49c None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:25.000232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:25.019204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686d2f5-04ed-4157-af5c-a075c3956ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:25.019817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d2f5-04ed-4157-af5c-a075c3956ada None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:25.020519 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d2f5-04ed-4157-af5c-a075c3956ada None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:25.020796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d2f5-04ed-4157-af5c-a075c3956ada None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:25.021413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686d2f5-04ed-4157-af5c-a075c3956ada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:25.023848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686d2f5-04ed-4157-af5c-a075c3956ada None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:25.030691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 393/761] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Oct 14 17:43:24 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:43:25.299703 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99811548-55cc-400d-b304-e34324caf10a None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:25.300109 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99811548-55cc-400d-b304-e34324caf10a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:25.300390 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99811548-55cc-400d-b304-e34324caf10a None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:25.301962 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 369/762] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:25.308192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940b916-25ab-481e-adf4-ebecb600526f None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:25.308478 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940b916-25ab-481e-adf4-ebecb600526f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:25.308755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940b916-25ab-481e-adf4-ebecb600526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:25.327595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9940b916-25ab-481e-adf4-ebecb600526f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:25.329133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9940b916-25ab-481e-adf4-ebecb600526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:25.399767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 394/763] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:25 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:25.664771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea92442-af14-4bac-9817-ee09d5f3a603 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:25.665265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea92442-af14-4bac-9817-ee09d5f3a603 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:25.665265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea92442-af14-4bac-9817-ee09d5f3a603 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:25.666654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 370/764] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:25.672062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf3b1-2df8-4e35-abaa-becb6dce4c77 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:25.672590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf3b1-2df8-4e35-abaa-becb6dce4c77 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:25.673095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf3b1-2df8-4e35-abaa-becb6dce4c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:25.677841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea92442-af14-4bac-9817-ee09d5f3a603 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:25.700349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e0cdf3b1-2df8-4e35-abaa-becb6dce4c77 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:25.701859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e0cdf3b1-2df8-4e35-abaa-becb6dce4c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:25.792349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 395/765] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:25 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:25.806813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea92442-af14-4bac-9817-ee09d5f3a603 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:25.842915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae392e-145a-4498-9d95-2d174b81b0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:25.843963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae392e-145a-4498-9d95-2d174b81b0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:25.844465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae392e-145a-4498-9d95-2d174b81b0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:25.845039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae392e-145a-4498-9d95-2d174b81b0e8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:25.845819 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae392e-145a-4498-9d95-2d174b81b0e8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:25.847384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae392e-145a-4498-9d95-2d174b81b0e8 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-0fae392e-145a-4498-9d95-2d174b81b0e8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:25.857252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:25.858288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 17:43:25.859134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:25.861819 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 371/766] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Oct 14 17:43:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:25.868240 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf3b1-2df8-4e35-abaa-becb6dce4c77 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:25.911820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:25.982157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf3b1-2df8-4e35-abaa-becb6dce4c77 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:26.018586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633349e-9744-4470-8ae1-b4e157c34094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:26.019196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633349e-9744-4470-8ae1-b4e157c34094 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:26.019783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633349e-9744-4470-8ae1-b4e157c34094 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:26.020131 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633349e-9744-4470-8ae1-b4e157c34094 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:26.020763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633349e-9744-4470-8ae1-b4e157c34094 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:26.021107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633349e-9744-4470-8ae1-b4e157c34094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:26.022499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633349e-9744-4470-8ae1-b4e157c34094 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:26.028428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 396/767] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:43:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:26.036934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae392e-145a-4498-9d95-2d174b81b0e8 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:26.040130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:26.061593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e91f4d-7913-48c8-bf20-423755aeac0d None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:26.061915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e91f4d-7913-48c8-bf20-423755aeac0d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:26.062140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e91f4d-7913-48c8-bf20-423755aeac0d None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:26.063807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 397/768] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:26.068905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a305f-2edf-4064-b2ec-a697f33ed19d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:26.069183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a305f-2edf-4064-b2ec-a697f33ed19d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:26.069458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a305f-2edf-4064-b2ec-a697f33ed19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:26.088568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-467a305f-2edf-4064-b2ec-a697f33ed19d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:26.089769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-467a305f-2edf-4064-b2ec-a697f33ed19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:26.153209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae392e-145a-4498-9d95-2d174b81b0e8 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:26.161575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 398/769] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:26 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:26.162229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a305f-2edf-4064-b2ec-a697f33ed19d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:26.198892 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:26.209582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8000e3-d80f-4445-a4d7-6b70475a059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:26.211280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8000e3-d80f-4445-a4d7-6b70475a059f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:26.211763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8000e3-d80f-4445-a4d7-6b70475a059f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:26.212272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8000e3-d80f-4445-a4d7-6b70475a059f None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:26.213244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8000e3-d80f-4445-a4d7-6b70475a059f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:26.214086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8000e3-d80f-4445-a4d7-6b70475a059f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:26.216200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8000e3-d80f-4445-a4d7-6b70475a059f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:26.226179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 372/770] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Oct 14 17:43:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:26.235714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8000e3-d80f-4445-a4d7-6b70475a059f None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:26.287202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a305f-2edf-4064-b2ec-a697f33ed19d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:26.307993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:26.326084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41092ec-ed7b-46e1-8e0f-d6e11211a76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:26.326796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41092ec-ed7b-46e1-8e0f-d6e11211a76b None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:26.327103 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41092ec-ed7b-46e1-8e0f-d6e11211a76b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:26.327444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41092ec-ed7b-46e1-8e0f-d6e11211a76b None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:26.328113 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41092ec-ed7b-46e1-8e0f-d6e11211a76b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:26.334005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41092ec-ed7b-46e1-8e0f-d6e11211a76b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:26.348074 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8000e3-d80f-4445-a4d7-6b70475a059f None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:26.351612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.sql.core [None req-c41092ec-ed7b-46e1-8e0f-d6e11211a76b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 14 17:43:26.351612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 14 17:43:26.351612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [parameters: {'user_id': 'f49811b09691424c8731adb139a686dc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 14 17:43:26.351612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81267) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-c41092ec-ed7b-46e1-8e0f-d6e11211a76b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application conn.query(q) Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result.read() Oct 14 17:43:26.359790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 17:43:26.360898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application next(self.gen) Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application yield resource Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application next(self.gen) Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise self.value Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application session.commit() Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 17:43:26.362022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.session.flush() Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application compat.raise_( Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 17:43:26.363066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 17:43:26.364127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application conn.query(q) Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result.read() Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 17:43:26.364978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f49811b09691424c8731adb139a686dc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:26.365791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 14 17:43:26.367368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 14 17:43:26.369347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 14 17:43:26.369347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 14 17:43:26.369347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 17:43:26.369347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 17:43:26.369347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 14 17:43:26.369347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:26.371993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 399/771] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Oct 14 17:43:26 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:26.377279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41092ec-ed7b-46e1-8e0f-d6e11211a76b None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:26.392672 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3235e39-9fec-4a5c-92fb-6656010a8d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:26.393332 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3235e39-9fec-4a5c-92fb-6656010a8d7a None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:26.393680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3235e39-9fec-4a5c-92fb-6656010a8d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:26.394000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3235e39-9fec-4a5c-92fb-6656010a8d7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:26.394603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3235e39-9fec-4a5c-92fb-6656010a8d7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:26.396138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3235e39-9fec-4a5c-92fb-6656010a8d7a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:26.420219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 373/772] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:43:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:26.465803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3235e39-9fec-4a5c-92fb-6656010a8d7a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:26.520772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:26.529536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41092ec-ed7b-46e1-8e0f-d6e11211a76b None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:26.579794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf32b82-fa8e-44fa-8c53-86a80c7cfb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:26.580833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf32b82-fa8e-44fa-8c53-86a80c7cfb32 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:26.581305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf32b82-fa8e-44fa-8c53-86a80c7cfb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:26.581767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf32b82-fa8e-44fa-8c53-86a80c7cfb32 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:26.590707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf32b82-fa8e-44fa-8c53-86a80c7cfb32 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:26.592639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf32b82-fa8e-44fa-8c53-86a80c7cfb32 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-abf32b82-fa8e-44fa-8c53-86a80c7cfb32 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:43:26.605906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:26.607053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:26.608249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:26.608249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:26.608249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:43:26.608249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:43:26.608249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:43:26.608249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:43:26.608249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:43:26.608249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:43:26.608249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 17:43:26.608249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:26.610082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 400/773] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:43:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:26.611520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3235e39-9fec-4a5c-92fb-6656010a8d7a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:26.616682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf32b82-fa8e-44fa-8c53-86a80c7cfb32 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:26.642458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:26.658514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec449-f7a8-4b04-9fee-0322fd21592d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:26.659501 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec449-f7a8-4b04-9fee-0322fd21592d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:26.660069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec449-f7a8-4b04-9fee-0322fd21592d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:26.660532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec449-f7a8-4b04-9fee-0322fd21592d None admin] PATH_INFO: `/v3/services/image` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:26.661302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec449-f7a8-4b04-9fee-0322fd21592d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:26.662894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec449-f7a8-4b04-9fee-0322fd21592d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-387ec449-f7a8-4b04-9fee-0322fd21592d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:26.678357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:26.680861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 17:43:26.681709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:26.684176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 374/774] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Oct 14 17:43:26 2022] GET /identity/v3/services/image => generated 86 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:26.686526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec449-f7a8-4b04-9fee-0322fd21592d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:26.745304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf32b82-fa8e-44fa-8c53-86a80c7cfb32 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:26.782048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92348b30-0ac3-4fd1-8907-96cd856a4f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:26.782666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92348b30-0ac3-4fd1-8907-96cd856a4f06 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:26.782998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92348b30-0ac3-4fd1-8907-96cd856a4f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:26.783396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92348b30-0ac3-4fd1-8907-96cd856a4f06 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:26.784371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92348b30-0ac3-4fd1-8907-96cd856a4f06 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:26.784732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92348b30-0ac3-4fd1-8907-96cd856a4f06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:26.786386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92348b30-0ac3-4fd1-8907-96cd856a4f06 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:26.805794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec449-f7a8-4b04-9fee-0322fd21592d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:26.836237 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 401/775] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Oct 14 17:43:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:26.845552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f9f53-0d1f-4784-b626-f24ada6a8f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:26.846185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f9f53-0d1f-4784-b626-f24ada6a8f1a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:26.846461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f9f53-0d1f-4784-b626-f24ada6a8f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:26.846734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f9f53-0d1f-4784-b626-f24ada6a8f1a None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:26.847381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f9f53-0d1f-4784-b626-f24ada6a8f1a None admin] RBAC: Adding query filter params (name=image) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:26.847683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f9f53-0d1f-4784-b626-f24ada6a8f1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:26.849118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f9f53-0d1f-4784-b626-f24ada6a8f1a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:26.856132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 375/776] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:26.861900 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92348b30-0ac3-4fd1-8907-96cd856a4f06 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:26.880853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:26.984973 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92348b30-0ac3-4fd1-8907-96cd856a4f06 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:27.026703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d53128-c31d-4187-912c-ac8a42bb0c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:27.027310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d53128-c31d-4187-912c-ac8a42bb0c59 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:27.027580 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d53128-c31d-4187-912c-ac8a42bb0c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:27.027843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d53128-c31d-4187-912c-ac8a42bb0c59 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:27.028499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d53128-c31d-4187-912c-ac8a42bb0c59 None admin] RBAC: Adding query filter params (type=image) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:27.028793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d53128-c31d-4187-912c-ac8a42bb0c59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:27.030221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d53128-c31d-4187-912c-ac8a42bb0c59 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:27.036715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 402/777] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:27.042456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f9f53-0d1f-4784-b626-f24ada6a8f1a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:27.101969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:27.148702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f9f53-0d1f-4784-b626-f24ada6a8f1a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:27.185396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113af69-8e0b-4d97-af9f-99b7006946ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:27.186031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113af69-8e0b-4d97-af9f-99b7006946ef None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:27.186303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113af69-8e0b-4d97-af9f-99b7006946ef None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:27.186574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113af69-8e0b-4d97-af9f-99b7006946ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:27.187270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113af69-8e0b-4d97-af9f-99b7006946ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:27.188718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113af69-8e0b-4d97-af9f-99b7006946ef None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:27.215496 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 376/778] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:43:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:27.309695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392259c1-1d36-44d7-88f5-50c5ddc99b76 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:27.310034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392259c1-1d36-44d7-88f5-50c5ddc99b76 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:27.310244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392259c1-1d36-44d7-88f5-50c5ddc99b76 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:27.311824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 403/779] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:27.317322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcc550-49ee-4341-a5ae-1c53719360b3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:27.317596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcc550-49ee-4341-a5ae-1c53719360b3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:27.317867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcc550-49ee-4341-a5ae-1c53719360b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:27.337213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-8fdcc550-49ee-4341-a5ae-1c53719360b3 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:27.338425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8fdcc550-49ee-4341-a5ae-1c53719360b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:27.421178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 377/780] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:27 2022] POST /identity/v3/auth/tokens => generated 1962 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:27.693277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392259c1-1d36-44d7-88f5-50c5ddc99b76 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:27.772688 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:27.799560 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392259c1-1d36-44d7-88f5-50c5ddc99b76 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:27.834667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a5c55-76bf-4d51-bd11-3f058edf8217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:27.835320 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a5c55-76bf-4d51-bd11-3f058edf8217 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:27.835594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a5c55-76bf-4d51-bd11-3f058edf8217 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:27.835862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a5c55-76bf-4d51-bd11-3f058edf8217 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:27.836468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a5c55-76bf-4d51-bd11-3f058edf8217 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:27.837832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a5c55-76bf-4d51-bd11-3f058edf8217 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-ad3a5c55-76bf-4d51-bd11-3f058edf8217 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:27.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:27.847590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:27.848481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 404/781] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:27.853564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcc550-49ee-4341-a5ae-1c53719360b3 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:27.888337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:27.956143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcc550-49ee-4341-a5ae-1c53719360b3 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:27.990963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b67c5-3da9-4e4c-8c16-db5a14da0a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:27.991706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b67c5-3da9-4e4c-8c16-db5a14da0a72 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:27.992013 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b67c5-3da9-4e4c-8c16-db5a14da0a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:27.992285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b67c5-3da9-4e4c-8c16-db5a14da0a72 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:27.992909 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b67c5-3da9-4e4c-8c16-db5a14da0a72 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:27.993205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b67c5-3da9-4e4c-8c16-db5a14da0a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:27.994607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b67c5-3da9-4e4c-8c16-db5a14da0a72 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:28.004217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 378/782] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:43:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:28.005438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a5c55-76bf-4d51-bd11-3f058edf8217 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:28.110584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a5c55-76bf-4d51-bd11-3f058edf8217 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:28.146300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9483cf-de91-4742-b018-f6affc2f2ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:28.146909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9483cf-de91-4742-b018-f6affc2f2ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:28.147249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9483cf-de91-4742-b018-f6affc2f2ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:28.147520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9483cf-de91-4742-b018-f6affc2f2ff4 None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:28.148746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9483cf-de91-4742-b018-f6affc2f2ff4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:28.148928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9483cf-de91-4742-b018-f6affc2f2ff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:28.150243 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9483cf-de91-4742-b018-f6affc2f2ff4 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:28.159947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 405/783] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Oct 14 17:43:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:28.161467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b67c5-3da9-4e4c-8c16-db5a14da0a72 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:28.261729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:28.265564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b67c5-3da9-4e4c-8c16-db5a14da0a72 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:28.300737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa37c67-c591-45a6-ae00-b8990b668977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:28.301326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37c67-c591-45a6-ae00-b8990b668977 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:28.301596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37c67-c591-45a6-ae00-b8990b668977 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:28.301860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37c67-c591-45a6-ae00-b8990b668977 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:28.302522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa37c67-c591-45a6-ae00-b8990b668977 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:28.303859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa37c67-c591-45a6-ae00-b8990b668977 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:28.324463 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 379/784] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:43:28 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:28.705787 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827ec82-2046-418d-b0ae-b47e910d1346 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:28.706180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827ec82-2046-418d-b0ae-b47e910d1346 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:28.706311 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827ec82-2046-418d-b0ae-b47e910d1346 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:28.707782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 406/785] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:28.713037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb62da4-8638-4863-9a64-ca4f2431fc9d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:28.713324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb62da4-8638-4863-9a64-ca4f2431fc9d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:28.713594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb62da4-8638-4863-9a64-ca4f2431fc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:28.733505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2bb62da4-8638-4863-9a64-ca4f2431fc9d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:28.734678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2bb62da4-8638-4863-9a64-ca4f2431fc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:28.809428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 380/786] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:29.078503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827ec82-2046-418d-b0ae-b47e910d1346 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:29.081784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:29.185025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827ec82-2046-418d-b0ae-b47e910d1346 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:29.219067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80051abf-7f14-4ac6-8679-ed9877134368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:29.219687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80051abf-7f14-4ac6-8679-ed9877134368 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:29.220000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80051abf-7f14-4ac6-8679-ed9877134368 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:29.220275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80051abf-7f14-4ac6-8679-ed9877134368 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:29.221386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80051abf-7f14-4ac6-8679-ed9877134368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:29.222827 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80051abf-7f14-4ac6-8679-ed9877134368 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:29.227921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 407/787] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:29.230794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb62da4-8638-4863-9a64-ca4f2431fc9d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:29.315106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:29.336025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb62da4-8638-4863-9a64-ca4f2431fc9d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:29.375704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28aae46-8444-4dff-84e3-6585de8cd08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:29.376341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28aae46-8444-4dff-84e3-6585de8cd08d None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:29.376621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28aae46-8444-4dff-84e3-6585de8cd08d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:29.376890 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28aae46-8444-4dff-84e3-6585de8cd08d None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:29.377556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28aae46-8444-4dff-84e3-6585de8cd08d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:29.378975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28aae46-8444-4dff-84e3-6585de8cd08d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:29.409467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 381/788] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:43:29 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:29.466907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90ff30-50f0-4032-8742-5fb4bb45e386 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:29.467229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90ff30-50f0-4032-8742-5fb4bb45e386 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:29.467452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90ff30-50f0-4032-8742-5fb4bb45e386 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:29.469026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 408/789] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:29.474706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a987bd5-6766-4589-bc9f-0ffd01dc7235 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:29.475007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a987bd5-6766-4589-bc9f-0ffd01dc7235 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:29.475261 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a987bd5-6766-4589-bc9f-0ffd01dc7235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:29.496957 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-8a987bd5-6766-4589-bc9f-0ffd01dc7235 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:29.498154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8a987bd5-6766-4589-bc9f-0ffd01dc7235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:29.572537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 382/790] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:29.843562 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90ff30-50f0-4032-8742-5fb4bb45e386 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:29.912180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:29.945993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90ff30-50f0-4032-8742-5fb4bb45e386 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:29.981356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037994-2a48-4dd0-a3de-578815148447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:29.981961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037994-2a48-4dd0-a3de-578815148447 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:29.982236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037994-2a48-4dd0-a3de-578815148447 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:29.982502 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037994-2a48-4dd0-a3de-578815148447 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:29.983169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e037994-2a48-4dd0-a3de-578815148447 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:29.984575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e037994-2a48-4dd0-a3de-578815148447 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:29.993106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.sql.core [None req-6e037994-2a48-4dd0-a3de-578815148447 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 17:43:29.993106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 17:43:29.993106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [parameters: {'id': 'b7a40343b1714806be82209d6a76d3de', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 17:43:29.993106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81267) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-6e037994-2a48-4dd0-a3de-578815148447 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application conn.query(q) Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 17:43:30.000483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result.read() Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 14 17:43:30.001336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application next(self.gen) Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application yield resource Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application next(self.gen) Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise self.value Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application session.commit() Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 17:43:30.002309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.session.flush() Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application compat.raise_( Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 17:43:30.003204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 17:43:30.004862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application conn.query(q) Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application result.read() Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 17:43:30.007010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [parameters: {'id': 'b7a40343b1714806be82209d6a76d3de', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 14 17:43:30.009677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:30.011486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:30.017111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 409/791] 10.222.0.27 () {60 vars in 1137 bytes} [Fri Oct 14 17:43:29 2022] POST /identity/v3/domains => generated 190 bytes in 163 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:30.018712 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a987bd5-6766-4589-bc9f-0ffd01dc7235 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:30.081924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:30.123364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a987bd5-6766-4589-bc9f-0ffd01dc7235 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:30.158282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf7703-5eb6-45c7-a144-a3da7e8af300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:30.158912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf7703-5eb6-45c7-a144-a3da7e8af300 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:30.159186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf7703-5eb6-45c7-a144-a3da7e8af300 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:30.159454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf7703-5eb6-45c7-a144-a3da7e8af300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:30.170422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf7703-5eb6-45c7-a144-a3da7e8af300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:30.171942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf7703-5eb6-45c7-a144-a3da7e8af300 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-1faf7703-5eb6-45c7-a144-a3da7e8af300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:30.186593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:30.187579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:30.193887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:30.193887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:30.193887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:30.193887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 383/792] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:30.195970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037994-2a48-4dd0-a3de-578815148447 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:30.300450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037994-2a48-4dd0-a3de-578815148447 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:30.334515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0c647-79df-45e3-838c-e7d78ac0bd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:30.335962 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0c647-79df-45e3-838c-e7d78ac0bd12 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:30.336337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0c647-79df-45e3-838c-e7d78ac0bd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:30.336734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0c647-79df-45e3-838c-e7d78ac0bd12 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:30.337503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0c647-79df-45e3-838c-e7d78ac0bd12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:30.337905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0c647-79df-45e3-838c-e7d78ac0bd12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:30.339414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0c647-79df-45e3-838c-e7d78ac0bd12 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:30.341415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:30.363911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 410/793] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:30.587311 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bf6e7-330d-4516-898f-dbaf409055df None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:30.587858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bf6e7-330d-4516-898f-dbaf409055df None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:30.587858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bf6e7-330d-4516-898f-dbaf409055df None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:30.589336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 384/794] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:30.594395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8c12-15ab-4394-be0d-d31d7b70434c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:30.594680 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8c12-15ab-4394-be0d-d31d7b70434c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:30.594959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8c12-15ab-4394-be0d-d31d7b70434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:30.613680 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ab9a8c12-15ab-4394-be0d-d31d7b70434c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:30.615188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ab9a8c12-15ab-4394-be0d-d31d7b70434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:30.685699 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 411/795] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:30.957799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bf6e7-330d-4516-898f-dbaf409055df None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:31.059361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bf6e7-330d-4516-898f-dbaf409055df None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:31.076066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:31.094355 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1213f-8511-47fd-9337-c59d531d07f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:31.094974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1213f-8511-47fd-9337-c59d531d07f6 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:31.095250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1213f-8511-47fd-9337-c59d531d07f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:31.095517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1213f-8511-47fd-9337-c59d531d07f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:31.105844 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1213f-8511-47fd-9337-c59d531d07f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-9df1213f-8511-47fd-9337-c59d531d07f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:43:31.110885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:31.112023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:31.113132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 385/796] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Oct 14 17:43:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:31.118526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a8c12-15ab-4394-be0d-d31d7b70434c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:31.214284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a8c12-15ab-4394-be0d-d31d7b70434c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:31.237328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:31.248999 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efda18-0cc0-4fd5-934b-7fac2de1c549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:31.249607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efda18-0cc0-4fd5-934b-7fac2de1c549 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:31.249884 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efda18-0cc0-4fd5-934b-7fac2de1c549 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:31.250152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efda18-0cc0-4fd5-934b-7fac2de1c549 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:31.250807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efda18-0cc0-4fd5-934b-7fac2de1c549 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:31.251128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efda18-0cc0-4fd5-934b-7fac2de1c549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:31.252544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efda18-0cc0-4fd5-934b-7fac2de1c549 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:31.260425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 412/797] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Oct 14 17:43:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:31.266505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1213f-8511-47fd-9337-c59d531d07f6 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:31.368385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1213f-8511-47fd-9337-c59d531d07f6 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:31.404944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21886b39-7671-4349-b410-8ffe7aa42ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:31.405562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21886b39-7671-4349-b410-8ffe7aa42ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:31.405836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21886b39-7671-4349-b410-8ffe7aa42ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:31.406090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21886b39-7671-4349-b410-8ffe7aa42ab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:31.418208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21886b39-7671-4349-b410-8ffe7aa42ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:31.419632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21886b39-7671-4349-b410-8ffe7aa42ab0 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-21886b39-7671-4349-b410-8ffe7aa42ab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:31.434259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:31.435428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:31.436423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:31.436423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:31.436423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:31.440033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 386/798] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:31.441393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efda18-0cc0-4fd5-934b-7fac2de1c549 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:31.558017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efda18-0cc0-4fd5-934b-7fac2de1c549 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:31.599209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276f317-23b9-4752-b831-bfea007a5bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:31.600223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276f317-23b9-4752-b831-bfea007a5bac None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:31.600676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276f317-23b9-4752-b831-bfea007a5bac None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:31.601757 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276f317-23b9-4752-b831-bfea007a5bac None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:31.602865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276f317-23b9-4752-b831-bfea007a5bac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:31.603386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276f317-23b9-4752-b831-bfea007a5bac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:31.605907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276f317-23b9-4752-b831-bfea007a5bac None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:31.620101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 413/799] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:31.625769 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21886b39-7671-4349-b410-8ffe7aa42ab0 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:31.646343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2b8bb-57cf-4d4e-8d10-460be8d33165 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:31.646625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2b8bb-57cf-4d4e-8d10-460be8d33165 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:31.646902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2b8bb-57cf-4d4e-8d10-460be8d33165 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:31.648805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 414/800] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:43:31.654323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651ec9a-a14b-4cf3-ac5c-30f2c4a00796 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:31.654603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651ec9a-a14b-4cf3-ac5c-30f2c4a00796 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:31.654883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651ec9a-a14b-4cf3-ac5c-30f2c4a00796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:31.657960 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:31.674027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9651ec9a-a14b-4cf3-ac5c-30f2c4a00796 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:31.675221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9651ec9a-a14b-4cf3-ac5c-30f2c4a00796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:31.728742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:31.737983 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21886b39-7671-4349-b410-8ffe7aa42ab0 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:31.754596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 415/801] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:31.778406 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4c878-9d83-4667-9c5a-42658bfb5f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:31.779040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4c878-9d83-4667-9c5a-42658bfb5f8a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:31.779315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4c878-9d83-4667-9c5a-42658bfb5f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:31.779583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4c878-9d83-4667-9c5a-42658bfb5f8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:31.787280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4c878-9d83-4667-9c5a-42658bfb5f8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:31.788815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4c878-9d83-4667-9c5a-42658bfb5f8a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-49e4c878-9d83-4667-9c5a-42658bfb5f8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:43:31.801650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:31.802833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:31.803903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 387/802] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Oct 14 17:43:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:31.810262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651ec9a-a14b-4cf3-ac5c-30f2c4a00796 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:31.926625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651ec9a-a14b-4cf3-ac5c-30f2c4a00796 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:31.963657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fe481-dede-4afb-a20d-28ed12795912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:31.964292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fe481-dede-4afb-a20d-28ed12795912 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:31.964569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fe481-dede-4afb-a20d-28ed12795912 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:31.964835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fe481-dede-4afb-a20d-28ed12795912 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:31.965603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fe481-dede-4afb-a20d-28ed12795912 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:31.965903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fe481-dede-4afb-a20d-28ed12795912 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:31.967323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fe481-dede-4afb-a20d-28ed12795912 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:31.985163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 416/803] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Oct 14 17:43:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:31.990891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4c878-9d83-4667-9c5a-42658bfb5f8a None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:32.081523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:32.087730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fe481-dede-4afb-a20d-28ed12795912 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:32.097371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4c878-9d83-4667-9c5a-42658bfb5f8a None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:32.124518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:32.138696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70d227-730e-4a92-b659-493164fd34a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:32.139511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70d227-730e-4a92-b659-493164fd34a4 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:32.140071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70d227-730e-4a92-b659-493164fd34a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:32.140343 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70d227-730e-4a92-b659-493164fd34a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:32.153701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70d227-730e-4a92-b659-493164fd34a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:32.155266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70d227-730e-4a92-b659-493164fd34a4 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-9c70d227-730e-4a92-b659-493164fd34a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:32.170150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:32.171289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:32.172354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:32.172354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:32.172354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:32.172354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 388/804] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:32.177210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70d227-730e-4a92-b659-493164fd34a4 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:32.204388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fe481-dede-4afb-a20d-28ed12795912 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:32.238733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b99e3-3eb5-4bcb-bb0a-34478e618973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:32.239395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b99e3-3eb5-4bcb-bb0a-34478e618973 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:32.239755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b99e3-3eb5-4bcb-bb0a-34478e618973 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:32.240072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b99e3-3eb5-4bcb-bb0a-34478e618973 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:32.241246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b99e3-3eb5-4bcb-bb0a-34478e618973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:32.243781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b99e3-3eb5-4bcb-bb0a-34478e618973 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:32.245393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 417/805] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:32.251503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b99e3-3eb5-4bcb-bb0a-34478e618973 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:32.296513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70d227-730e-4a92-b659-493164fd34a4 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:32.335104 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb7d37-0399-45e2-890d-d726044267ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:32.335808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb7d37-0399-45e2-890d-d726044267ad None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:32.336304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb7d37-0399-45e2-890d-d726044267ad None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:32.336585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb7d37-0399-45e2-890d-d726044267ad None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:32.337227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb7d37-0399-45e2-890d-d726044267ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:32.337550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb7d37-0399-45e2-890d-d726044267ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:32.338911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb7d37-0399-45e2-890d-d726044267ad None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:32.348720 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:32.353007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 389/806] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:32.358283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb7d37-0399-45e2-890d-d726044267ad None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:32.373706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b99e3-3eb5-4bcb-bb0a-34478e618973 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:32.375667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:32.422089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef974dd9-0b25-4e09-a0ee-cc55d8b3edcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:32.422881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef974dd9-0b25-4e09-a0ee-cc55d8b3edcf None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:32.423369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef974dd9-0b25-4e09-a0ee-cc55d8b3edcf None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:32.423838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef974dd9-0b25-4e09-a0ee-cc55d8b3edcf None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:32.424822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef974dd9-0b25-4e09-a0ee-cc55d8b3edcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:32.426317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef974dd9-0b25-4e09-a0ee-cc55d8b3edcf None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:32.449642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 418/807] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:43:32 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:43:32.481279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb7d37-0399-45e2-890d-d726044267ad None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:32.518832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71a6ae-3aaa-485b-9855-bde1ee49f892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:32.519469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71a6ae-3aaa-485b-9855-bde1ee49f892 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:32.519745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71a6ae-3aaa-485b-9855-bde1ee49f892 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:32.520053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71a6ae-3aaa-485b-9855-bde1ee49f892 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:32.527551 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71a6ae-3aaa-485b-9855-bde1ee49f892 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:32.529129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71a6ae-3aaa-485b-9855-bde1ee49f892 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-2c71a6ae-3aaa-485b-9855-bde1ee49f892 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:43:32.541980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:32.542993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:32.543963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:43:32.543963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:43:32.543963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:43:32.543963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:43:32.543963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:43:32.543963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:43:32.543963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:32.543963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:32.543963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 390/808] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:32.549160 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef974dd9-0b25-4e09-a0ee-cc55d8b3edcf None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:32.658281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef974dd9-0b25-4e09-a0ee-cc55d8b3edcf None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:32.692891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14506fa6-9f6a-473d-96ab-c0d1f05281c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:32.693505 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14506fa6-9f6a-473d-96ab-c0d1f05281c7 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:32.693790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14506fa6-9f6a-473d-96ab-c0d1f05281c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:32.694103 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14506fa6-9f6a-473d-96ab-c0d1f05281c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:32.694850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14506fa6-9f6a-473d-96ab-c0d1f05281c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:32.695158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14506fa6-9f6a-473d-96ab-c0d1f05281c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:32.696646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14506fa6-9f6a-473d-96ab-c0d1f05281c7 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:32.708739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 419/809] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:32.715091 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71a6ae-3aaa-485b-9855-bde1ee49f892 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:32.774303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:32.815806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71a6ae-3aaa-485b-9855-bde1ee49f892 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:32.849296 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f89bd-9d80-4d90-9f89-d22a6196a5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:32.849955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f89bd-9d80-4d90-9f89-d22a6196a5c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:32.850229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f89bd-9d80-4d90-9f89-d22a6196a5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:32.850498 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f89bd-9d80-4d90-9f89-d22a6196a5c9 None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/e42968016de8470c933655e921fb3394/roles/70d2eb11714847ce88bb184a459377d9` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:32.853272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f89bd-9d80-4d90-9f89-d22a6196a5c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=e42968016de8470c933655e921fb3394, role_id=70d2eb11714847ce88bb184a459377d9)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:32.854694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f89bd-9d80-4d90-9f89-d22a6196a5c9 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:32.867292 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 391/810] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:43:32 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/e42968016de8470c933655e921fb3394/roles/70d2eb11714847ce88bb184a459377d9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:43:34.679943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125a94c-5fac-4967-b771-6e55653afc99 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:34.680458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125a94c-5fac-4967-b771-6e55653afc99 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:34.680458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125a94c-5fac-4967-b771-6e55653afc99 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:34.681734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 420/811] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:34.687723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68737484-2cbc-4f51-a750-d8d5fc0cc40e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:34.688172 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68737484-2cbc-4f51-a750-d8d5fc0cc40e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:34.688439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68737484-2cbc-4f51-a750-d8d5fc0cc40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:34.706771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-68737484-2cbc-4f51-a750-d8d5fc0cc40e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:34.707955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-68737484-2cbc-4f51-a750-d8d5fc0cc40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:34.774873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 392/812] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:35.002301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fa891-b49c-4939-9adf-0722c40dafe6 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:35.002593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fa891-b49c-4939-9adf-0722c40dafe6 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:35.002936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fa891-b49c-4939-9adf-0722c40dafe6 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:35.004366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 421/813] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:35.010258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c39a5-ce8a-4560-a469-bb2475b13bf3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:35.010537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c39a5-ce8a-4560-a469-bb2475b13bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:35.010805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c39a5-ce8a-4560-a469-bb2475b13bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:35.029689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-4d1c39a5-ce8a-4560-a469-bb2475b13bf3 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:35.030848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-4d1c39a5-ce8a-4560-a469-bb2475b13bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:35.054544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fa891-b49c-4939-9adf-0722c40dafe6 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:35.072881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:35.078786 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:35.099676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 393/814] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:35.156059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fa891-b49c-4939-9adf-0722c40dafe6 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:35.187900 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09905257-2226-44e1-a047-2b90f98e8ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:35.188536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09905257-2226-44e1-a047-2b90f98e8ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:35.188810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09905257-2226-44e1-a047-2b90f98e8ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:35.189074 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09905257-2226-44e1-a047-2b90f98e8ac4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:35.199338 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09905257-2226-44e1-a047-2b90f98e8ac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-09905257-2226-44e1-a047-2b90f98e8ac4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:43:35.204250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:35.205387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:35.206286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 422/815] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:43:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:35.211923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c39a5-ce8a-4560-a469-bb2475b13bf3 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:35.308798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c39a5-ce8a-4560-a469-bb2475b13bf3 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:35.341955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ec8c3-93d1-4ea3-ac77-8ee2904070a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:35.342544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ec8c3-93d1-4ea3-ac77-8ee2904070a1 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:35.342812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ec8c3-93d1-4ea3-ac77-8ee2904070a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:35.343075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ec8c3-93d1-4ea3-ac77-8ee2904070a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:35.343717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ec8c3-93d1-4ea3-ac77-8ee2904070a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:35.344039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ec8c3-93d1-4ea3-ac77-8ee2904070a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:35.345382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ec8c3-93d1-4ea3-ac77-8ee2904070a1 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:35.355863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 394/816] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:35.361728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09905257-2226-44e1-a047-2b90f98e8ac4 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:35.370029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ec8c3-93d1-4ea3-ac77-8ee2904070a1 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:35.416246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:35.442143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:35.465528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09905257-2226-44e1-a047-2b90f98e8ac4 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:35.503026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a6aaf-915c-48ff-8b82-81d56568a38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:35.503660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6aaf-915c-48ff-8b82-81d56568a38f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:35.503959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6aaf-915c-48ff-8b82-81d56568a38f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:35.504225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6aaf-915c-48ff-8b82-81d56568a38f None admin] PATH_INFO: `/v3/users/505771b2e47f4727b7c93e33c4a92acc` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:35.519960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a6aaf-915c-48ff-8b82-81d56568a38f None admin] RBAC: Authorizing `identity:get_user(user_id=505771b2e47f4727b7c93e33c4a92acc)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:35.521423 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a6aaf-915c-48ff-8b82-81d56568a38f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:35.524076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 423/817] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:43:35 2022] GET /identity/v3/users/505771b2e47f4727b7c93e33c4a92acc => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:35.530116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a6aaf-915c-48ff-8b82-81d56568a38f None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:35.586522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ec8c3-93d1-4ea3-ac77-8ee2904070a1 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:35.620392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b7fe2-4e03-4eb5-862d-6315e3e4f122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:35.621002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b7fe2-4e03-4eb5-862d-6315e3e4f122 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:35.621274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b7fe2-4e03-4eb5-862d-6315e3e4f122 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:35.621538 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b7fe2-4e03-4eb5-862d-6315e3e4f122 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:35.631274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b7fe2-4e03-4eb5-862d-6315e3e4f122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:43:35.633683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a6aaf-915c-48ff-8b82-81d56568a38f None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-071b7fe2-4e03-4eb5-862d-6315e3e4f122 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:43:35.636349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:35.637276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:35.638257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 395/818] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Oct 14 17:43:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:43:35.643605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b7fe2-4e03-4eb5-862d-6315e3e4f122 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:35.667342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b88bb-2fd0-4bb9-a8f3-06de54e416a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:35.667991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b88bb-2fd0-4bb9-a8f3-06de54e416a5 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:35.668264 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b88bb-2fd0-4bb9-a8f3-06de54e416a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:35.668534 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b88bb-2fd0-4bb9-a8f3-06de54e416a5 None admin] PATH_INFO: `/v3/projects/cbe42d45af6a44d0b684f98377f31cbc` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:35.669620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b88bb-2fd0-4bb9-a8f3-06de54e416a5 None admin] RBAC: Authorizing `identity:get_project(project_id=cbe42d45af6a44d0b684f98377f31cbc)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:35.671054 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b88bb-2fd0-4bb9-a8f3-06de54e416a5 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:35.672688 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 424/819] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:43:35 2022] GET /identity/v3/projects/cbe42d45af6a44d0b684f98377f31cbc => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:35.678553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b88bb-2fd0-4bb9-a8f3-06de54e416a5 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:35.682518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:35.745033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b7fe2-4e03-4eb5-862d-6315e3e4f122 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:35.746530 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:35.778802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b88bb-2fd0-4bb9-a8f3-06de54e416a5 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:35.781134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ab69d-299a-452a-b36b-ef606e4a8a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:35.781757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ab69d-299a-452a-b36b-ef606e4a8a64 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:35.782035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ab69d-299a-452a-b36b-ef606e4a8a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:35.782302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ab69d-299a-452a-b36b-ef606e4a8a64 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:35.782953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ab69d-299a-452a-b36b-ef606e4a8a64 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:35.783249 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ab69d-299a-452a-b36b-ef606e4a8a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:35.784624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ab69d-299a-452a-b36b-ef606e4a8a64 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:35.797491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 396/820] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Oct 14 17:43:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:35.798457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ab69d-299a-452a-b36b-ef606e4a8a64 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:35.812140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4803a2-2227-4e89-8059-fff2e82db9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:35.812788 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4803a2-2227-4e89-8059-fff2e82db9bd None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:35.813174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4803a2-2227-4e89-8059-fff2e82db9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:35.813446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4803a2-2227-4e89-8059-fff2e82db9bd None admin] PATH_INFO: `/v3/projects/cbe42d45af6a44d0b684f98377f31cbc/users/505771b2e47f4727b7c93e33c4a92acc/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:35.815336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4803a2-2227-4e89-8059-fff2e82db9bd None admin] RBAC: Authorizing `identity:create_grant(project_id=cbe42d45af6a44d0b684f98377f31cbc, user_id=505771b2e47f4727b7c93e33c4a92acc, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:35.816800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4803a2-2227-4e89-8059-fff2e82db9bd None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:35.830928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 425/821] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:43:35 2022] PUT /identity/v3/projects/cbe42d45af6a44d0b684f98377f31cbc/users/505771b2e47f4727b7c93e33c4a92acc/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:43:35.905345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ab69d-299a-452a-b36b-ef606e4a8a64 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:35.941610 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3cc6f-95cf-4016-bba0-604d5e2925e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:35.942254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3cc6f-95cf-4016-bba0-604d5e2925e0 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:35.942531 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3cc6f-95cf-4016-bba0-604d5e2925e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:35.942803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3cc6f-95cf-4016-bba0-604d5e2925e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:35.953958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3cc6f-95cf-4016-bba0-604d5e2925e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:35.955447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3cc6f-95cf-4016-bba0-604d5e2925e0 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-fed3cc6f-95cf-4016-bba0-604d5e2925e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:35.971137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:35.972324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:35.973227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:35.973227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:35.973227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:35.973227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 397/822] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:35.979523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4803a2-2227-4e89-8059-fff2e82db9bd None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:36.046400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:36.084293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4803a2-2227-4e89-8059-fff2e82db9bd None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:36.117952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead5279-7236-47c8-b691-72ef9b119be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:36.118547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead5279-7236-47c8-b691-72ef9b119be6 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:36.118820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead5279-7236-47c8-b691-72ef9b119be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:36.119086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead5279-7236-47c8-b691-72ef9b119be6 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:36.119719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead5279-7236-47c8-b691-72ef9b119be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:36.120044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead5279-7236-47c8-b691-72ef9b119be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:36.121402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead5279-7236-47c8-b691-72ef9b119be6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:36.132409 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 426/823] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:36.137954 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3cc6f-95cf-4016-bba0-604d5e2925e0 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:36.157492 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:36.237266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3cc6f-95cf-4016-bba0-604d5e2925e0 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:36.270147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fb949-593b-4c47-8bdf-73cd30fcb4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:36.270765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fb949-593b-4c47-8bdf-73cd30fcb4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:36.271037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fb949-593b-4c47-8bdf-73cd30fcb4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:36.271305 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fb949-593b-4c47-8bdf-73cd30fcb4f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:36.279362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fb949-593b-4c47-8bdf-73cd30fcb4f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:36.280823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fb949-593b-4c47-8bdf-73cd30fcb4f1 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-738fb949-593b-4c47-8bdf-73cd30fcb4f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:43:36.292245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:36.298019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:36.299047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 398/824] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Oct 14 17:43:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:36.299683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead5279-7236-47c8-b691-72ef9b119be6 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:36.397255 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead5279-7236-47c8-b691-72ef9b119be6 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:36.421568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:36.432449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1f370-4f1a-4077-b331-778f8d432ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:36.433046 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1f370-4f1a-4077-b331-778f8d432ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:36.433321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1f370-4f1a-4077-b331-778f8d432ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:36.433588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1f370-4f1a-4077-b331-778f8d432ec9 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:36.434306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1f370-4f1a-4077-b331-778f8d432ec9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:36.434604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1f370-4f1a-4077-b331-778f8d432ec9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:36.436048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1f370-4f1a-4077-b331-778f8d432ec9 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:36.452824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 427/825] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Oct 14 17:43:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:36.458533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fb949-593b-4c47-8bdf-73cd30fcb4f1 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:36.534630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:36.557440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fb949-593b-4c47-8bdf-73cd30fcb4f1 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:36.590061 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62775e-7414-4f07-8a29-1c45e4e4862d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:36.590651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62775e-7414-4f07-8a29-1c45e4e4862d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:36.590922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62775e-7414-4f07-8a29-1c45e4e4862d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:36.591189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62775e-7414-4f07-8a29-1c45e4e4862d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:36.656524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62775e-7414-4f07-8a29-1c45e4e4862d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:36.657953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62775e-7414-4f07-8a29-1c45e4e4862d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:36.731900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:36.754781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 399/826] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:43:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:43:36.761173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1f370-4f1a-4077-b331-778f8d432ec9 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:36.857430 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1f370-4f1a-4077-b331-778f8d432ec9 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:37.006727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b7916-ebc5-4503-816d-3db7708c8175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:37.007812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7916-ebc5-4503-816d-3db7708c8175 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:37.008115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7916-ebc5-4503-816d-3db7708c8175 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:37.008381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7916-ebc5-4503-816d-3db7708c8175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:37.018322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b7916-ebc5-4503-816d-3db7708c8175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:37.019740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b7916-ebc5-4503-816d-3db7708c8175 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-ea0b7916-ebc5-4503-816d-3db7708c8175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:43:37.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:37.035465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:43:37.036421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:43:37.036421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:43:37.036421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:37.036421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 428/827] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:37.041354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62775e-7414-4f07-8a29-1c45e4e4862d None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:37.138055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62775e-7414-4f07-8a29-1c45e4e4862d None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:37.170852 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5020ffd-cb63-4a21-82a0-c6e0b22cd642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:37.171439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5020ffd-cb63-4a21-82a0-c6e0b22cd642 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:37.171714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5020ffd-cb63-4a21-82a0-c6e0b22cd642 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:37.172005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5020ffd-cb63-4a21-82a0-c6e0b22cd642 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:37.172632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5020ffd-cb63-4a21-82a0-c6e0b22cd642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:37.172926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5020ffd-cb63-4a21-82a0-c6e0b22cd642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:37.174266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5020ffd-cb63-4a21-82a0-c6e0b22cd642 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:37.186202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 400/828] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:37.191777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b7916-ebc5-4503-816d-3db7708c8175 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:37.225100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:37.289927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b7916-ebc5-4503-816d-3db7708c8175 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:37.321708 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435577a1-8228-4b24-80a7-411c7b4e7475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:37.322315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435577a1-8228-4b24-80a7-411c7b4e7475 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:37.322589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435577a1-8228-4b24-80a7-411c7b4e7475 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:37.322855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435577a1-8228-4b24-80a7-411c7b4e7475 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:37.329293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435577a1-8228-4b24-80a7-411c7b4e7475 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:37.330719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435577a1-8228-4b24-80a7-411c7b4e7475 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-435577a1-8228-4b24-80a7-411c7b4e7475 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:43:37.343413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:37.344369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:37.345268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:43:37.345268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:43:37.345268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:43:37.345268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:43:37.345268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:43:37.345268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:43:37.345268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:43:37.345268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:37.345268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 429/829] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:43:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:37.350995 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5020ffd-cb63-4a21-82a0-c6e0b22cd642 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:37.386326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:37.449573 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5020ffd-cb63-4a21-82a0-c6e0b22cd642 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:37.482157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44542f-8c6c-4f22-bb2d-2ef771bde2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:37.482755 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44542f-8c6c-4f22-bb2d-2ef771bde2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:37.483031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44542f-8c6c-4f22-bb2d-2ef771bde2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:37.483299 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44542f-8c6c-4f22-bb2d-2ef771bde2c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:37.484006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44542f-8c6c-4f22-bb2d-2ef771bde2c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:37.484312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44542f-8c6c-4f22-bb2d-2ef771bde2c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:37.485723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44542f-8c6c-4f22-bb2d-2ef771bde2c2 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:37.498086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 401/830] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:43:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:43:37.503988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435577a1-8228-4b24-80a7-411c7b4e7475 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:37.597716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:37.604755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435577a1-8228-4b24-80a7-411c7b4e7475 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:37.637485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195de91a-78dc-4b25-afd6-98a7a4c94be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:37.638074 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195de91a-78dc-4b25-afd6-98a7a4c94be2 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:37.638348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195de91a-78dc-4b25-afd6-98a7a4c94be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:37.638615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195de91a-78dc-4b25-afd6-98a7a4c94be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:37.639327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195de91a-78dc-4b25-afd6-98a7a4c94be2 None admin] RBAC: Adding query filter params (role.id=70d2eb11714847ce88bb184a459377d9, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=e42968016de8470c933655e921fb3394) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:37.639629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195de91a-78dc-4b25-afd6-98a7a4c94be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:37.641354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195de91a-78dc-4b25-afd6-98a7a4c94be2 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:37.647144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 430/831] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:43:37 2022] GET /identity/v3/role_assignments?user.id=e42968016de8470c933655e921fb3394&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=70d2eb11714847ce88bb184a459377d9 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:38.040267 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a41438-b957-4e71-8350-0d9157eada32 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:38.040789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a41438-b957-4e71-8350-0d9157eada32 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:38.040789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a41438-b957-4e71-8350-0d9157eada32 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:38.042092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 402/832] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:38.047404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbf1e8-0561-4a96-9a41-e1df3d28fa67 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:38.047833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbf1e8-0561-4a96-9a41-e1df3d28fa67 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:38.047967 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbf1e8-0561-4a96-9a41-e1df3d28fa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:38.066281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-5fbbf1e8-0561-4a96-9a41-e1df3d28fa67 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:38.067447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5fbbf1e8-0561-4a96-9a41-e1df3d28fa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:38.136216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 431/833] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:38.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a41438-b957-4e71-8350-0d9157eada32 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:38.510027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:38.517306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a41438-b957-4e71-8350-0d9157eada32 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:38.550873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130580bc-cffc-4066-8970-12933e7a45af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:38.551478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130580bc-cffc-4066-8970-12933e7a45af None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:38.551746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130580bc-cffc-4066-8970-12933e7a45af None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:38.552031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130580bc-cffc-4066-8970-12933e7a45af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:38.562148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130580bc-cffc-4066-8970-12933e7a45af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-130580bc-cffc-4066-8970-12933e7a45af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:43:38.566828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:38.567908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:43:38.568766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 403/834] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:43:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:38.574307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbf1e8-0561-4a96-9a41-e1df3d28fa67 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:38.659014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:38.671862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbf1e8-0561-4a96-9a41-e1df3d28fa67 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:38.704865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f005656-e436-4d59-a797-69042e4ba2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:38.705453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f005656-e436-4d59-a797-69042e4ba2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:38.705720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f005656-e436-4d59-a797-69042e4ba2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:38.705981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f005656-e436-4d59-a797-69042e4ba2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:38.706622 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f005656-e436-4d59-a797-69042e4ba2d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:38.706912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f005656-e436-4d59-a797-69042e4ba2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:38.708286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f005656-e436-4d59-a797-69042e4ba2d6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:38.716739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 432/835] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:43:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:38.722602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130580bc-cffc-4066-8970-12933e7a45af None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:38.820872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130580bc-cffc-4066-8970-12933e7a45af None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:38.853887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4303dc-c061-4ea9-a14e-afe415143c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:38.854493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4303dc-c061-4ea9-a14e-afe415143c64 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:38.854768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4303dc-c061-4ea9-a14e-afe415143c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:38.855036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4303dc-c061-4ea9-a14e-afe415143c64 None admin] PATH_INFO: `/v3/users/505771b2e47f4727b7c93e33c4a92acc` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:38.856160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4303dc-c061-4ea9-a14e-afe415143c64 None admin] RBAC: Authorizing `identity:get_user(user_id=505771b2e47f4727b7c93e33c4a92acc)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:38.857579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4303dc-c061-4ea9-a14e-afe415143c64 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:38.859089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 404/836] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:43:38 2022] GET /identity/v3/users/505771b2e47f4727b7c93e33c4a92acc => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:38.864867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f005656-e436-4d59-a797-69042e4ba2d6 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:38.961171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f005656-e436-4d59-a797-69042e4ba2d6 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:38.992286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823a9b-e587-45f2-af33-1344f78c1aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:38.992864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823a9b-e587-45f2-af33-1344f78c1aaa None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:38.993135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823a9b-e587-45f2-af33-1344f78c1aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:38.993398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823a9b-e587-45f2-af33-1344f78c1aaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:39.011217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:39.058061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b823a9b-e587-45f2-af33-1344f78c1aaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:39.059483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b823a9b-e587-45f2-af33-1344f78c1aaa None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:39.152272 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 433/837] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:43:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:43:39.158146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4303dc-c061-4ea9-a14e-afe415143c64 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:39.182593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:39.253985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4303dc-c061-4ea9-a14e-afe415143c64 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:39.287760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126c700-6dd5-4845-8743-f5f0e484180f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:39.288394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126c700-6dd5-4845-8743-f5f0e484180f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:39.289090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126c700-6dd5-4845-8743-f5f0e484180f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:39.289368 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126c700-6dd5-4845-8743-f5f0e484180f None admin] PATH_INFO: `/v3/projects/cbe42d45af6a44d0b684f98377f31cbc` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:39.290465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126c700-6dd5-4845-8743-f5f0e484180f None admin] RBAC: Authorizing `identity:get_project(project_id=cbe42d45af6a44d0b684f98377f31cbc)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:39.291871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126c700-6dd5-4845-8743-f5f0e484180f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:39.293467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 405/838] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:43:39 2022] GET /identity/v3/projects/cbe42d45af6a44d0b684f98377f31cbc => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:39.299466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823a9b-e587-45f2-af33-1344f78c1aaa None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:39.347342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:39.396415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823a9b-e587-45f2-af33-1344f78c1aaa None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:39.428869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae422c8e-6821-4a0b-875a-c8ae5f450ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:39.430070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae422c8e-6821-4a0b-875a-c8ae5f450ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:39.430347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae422c8e-6821-4a0b-875a-c8ae5f450ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:39.430617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae422c8e-6821-4a0b-875a-c8ae5f450ba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:39.431334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae422c8e-6821-4a0b-875a-c8ae5f450ba7 None admin] RBAC: Adding query filter params (role.id=45b67ba9922c4b16baae3930d3de1b6c, scope.project.id=cbe42d45af6a44d0b684f98377f31cbc, user.id=505771b2e47f4727b7c93e33c4a92acc) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:39.431640 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae422c8e-6821-4a0b-875a-c8ae5f450ba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:39.433109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae422c8e-6821-4a0b-875a-c8ae5f450ba7 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:39.438595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 434/839] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:43:39 2022] GET /identity/v3/role_assignments?user.id=505771b2e47f4727b7c93e33c4a92acc&scope.project.id=cbe42d45af6a44d0b684f98377f31cbc&role.id=45b67ba9922c4b16baae3930d3de1b6c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:41.581853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677a288-da15-4ccf-9848-b4a31543959c None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:41.582375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677a288-da15-4ccf-9848-b4a31543959c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:41.582375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677a288-da15-4ccf-9848-b4a31543959c None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:41.583692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 406/840] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:41.589634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c6c9-9e34-4a3c-90fe-6fda48060b65 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:41.590011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c6c9-9e34-4a3c-90fe-6fda48060b65 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:41.590171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c6c9-9e34-4a3c-90fe-6fda48060b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:41.608767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-cb89c6c9-9e34-4a3c-90fe-6fda48060b65 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:41.609873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cb89c6c9-9e34-4a3c-90fe-6fda48060b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:41.674269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 435/841] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:41.943802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677a288-da15-4ccf-9848-b4a31543959c None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:42.034501 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:42.045810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677a288-da15-4ccf-9848-b4a31543959c None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:42.079823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06e7fd-0e5f-4c6e-8701-5c673e63d192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:42.080465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06e7fd-0e5f-4c6e-8701-5c673e63d192 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:42.080739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06e7fd-0e5f-4c6e-8701-5c673e63d192 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:42.081007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06e7fd-0e5f-4c6e-8701-5c673e63d192 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:42.082092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06e7fd-0e5f-4c6e-8701-5c673e63d192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:42.083501 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06e7fd-0e5f-4c6e-8701-5c673e63d192 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:42.085062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 407/842] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:42.091545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89c6c9-9e34-4a3c-90fe-6fda48060b65 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:42.134943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:42.187162 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89c6c9-9e34-4a3c-90fe-6fda48060b65 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:42.220186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f0f82-e114-4d62-82e5-52bd393706d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:42.220716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0f82-e114-4d62-82e5-52bd393706d8 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:42.220988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0f82-e114-4d62-82e5-52bd393706d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:42.221254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0f82-e114-4d62-82e5-52bd393706d8 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:42.221891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f0f82-e114-4d62-82e5-52bd393706d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:42.223310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f0f82-e114-4d62-82e5-52bd393706d8 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:42.243480 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 436/843] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:43:42 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:44.390570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e16b31-e147-457f-9df0-10a4149b1f66 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:44.391098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e16b31-e147-457f-9df0-10a4149b1f66 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:44.391098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e16b31-e147-457f-9df0-10a4149b1f66 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:44.392447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 408/844] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:44.397765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee0e77-170e-4e6e-a99b-c4c0163d3c3b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:44.398181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee0e77-170e-4e6e-a99b-c4c0163d3c3b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:44.398514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee0e77-170e-4e6e-a99b-c4c0163d3c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:44.416709 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f0ee0e77-170e-4e6e-a99b-c4c0163d3c3b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:44.417877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f0ee0e77-170e-4e6e-a99b-c4c0163d3c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:44.481985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 437/845] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:44.762957 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e16b31-e147-457f-9df0-10a4149b1f66 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:44.861048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e16b31-e147-457f-9df0-10a4149b1f66 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:44.894168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4394580-5000-409a-85e3-95159bb99b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:44.894777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4394580-5000-409a-85e3-95159bb99b57 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:44.895051 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4394580-5000-409a-85e3-95159bb99b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:44.895318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4394580-5000-409a-85e3-95159bb99b57 None admin] PATH_INFO: `/v3/roles/1cc2ea78f84c4491967839cfead3fd26` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:44.896705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4394580-5000-409a-85e3-95159bb99b57 None admin] RBAC: Authorizing `identity:get_role(role_id=1cc2ea78f84c4491967839cfead3fd26)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:44.898101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4394580-5000-409a-85e3-95159bb99b57 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:44.899526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 409/846] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:43:44 2022] GET /identity/v3/roles/1cc2ea78f84c4491967839cfead3fd26 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:44.905260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee0e77-170e-4e6e-a99b-c4c0163d3c3b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:44.939413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:45.006870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee0e77-170e-4e6e-a99b-c4c0163d3c3b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:45.039993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d99857-a15b-4d06-8763-2d98a970dd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:45.040606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d99857-a15b-4d06-8763-2d98a970dd23 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:45.040917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d99857-a15b-4d06-8763-2d98a970dd23 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:45.041143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d99857-a15b-4d06-8763-2d98a970dd23 None admin] PATH_INFO: `/v3/users/fb7e99f8b8ae4b15ba9f81a34cae8b32` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:45.054917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d99857-a15b-4d06-8763-2d98a970dd23 None admin] RBAC: Authorizing `identity:get_user(user_id=fb7e99f8b8ae4b15ba9f81a34cae8b32)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:45.056394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d99857-a15b-4d06-8763-2d98a970dd23 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:45.057977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 438/847] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:43:44 2022] GET /identity/v3/users/fb7e99f8b8ae4b15ba9f81a34cae8b32 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:45.063803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4394580-5000-409a-85e3-95159bb99b57 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:45.090637 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:45.162886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4394580-5000-409a-85e3-95159bb99b57 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:45.195576 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ba7bb-9f2d-4a33-9928-cc2de1f0369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:45.196211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ba7bb-9f2d-4a33-9928-cc2de1f0369a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:45.196486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ba7bb-9f2d-4a33-9928-cc2de1f0369a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:45.196759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ba7bb-9f2d-4a33-9928-cc2de1f0369a None admin] PATH_INFO: `/v3/projects/cbe42d45af6a44d0b684f98377f31cbc` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:45.197853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ba7bb-9f2d-4a33-9928-cc2de1f0369a None admin] RBAC: Authorizing `identity:get_project(project_id=cbe42d45af6a44d0b684f98377f31cbc)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:45.199253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ba7bb-9f2d-4a33-9928-cc2de1f0369a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:45.200860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 410/848] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:43:45 2022] GET /identity/v3/projects/cbe42d45af6a44d0b684f98377f31cbc => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:45.206785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d99857-a15b-4d06-8763-2d98a970dd23 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:45.291971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:45.303526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d99857-a15b-4d06-8763-2d98a970dd23 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:45.334867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696cc2b-730e-4149-86c9-13b54d5cbf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:45.335500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696cc2b-730e-4149-86c9-13b54d5cbf88 None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:45.336372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696cc2b-730e-4149-86c9-13b54d5cbf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:45.336646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696cc2b-730e-4149-86c9-13b54d5cbf88 None admin] PATH_INFO: `/v3/projects/cbe42d45af6a44d0b684f98377f31cbc/users/fb7e99f8b8ae4b15ba9f81a34cae8b32/roles/1cc2ea78f84c4491967839cfead3fd26` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:45.338719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696cc2b-730e-4149-86c9-13b54d5cbf88 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbe42d45af6a44d0b684f98377f31cbc, user_id=fb7e99f8b8ae4b15ba9f81a34cae8b32, role_id=1cc2ea78f84c4491967839cfead3fd26)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:45.340145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696cc2b-730e-4149-86c9-13b54d5cbf88 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:45.352626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 439/849] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:43:45 2022] PUT /identity/v3/projects/cbe42d45af6a44d0b684f98377f31cbc/users/fb7e99f8b8ae4b15ba9f81a34cae8b32/roles/1cc2ea78f84c4491967839cfead3fd26 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:43:47.490263 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b69fa-920e-4f96-87dc-a81efd90a520 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:47.490780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b69fa-920e-4f96-87dc-a81efd90a520 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:47.490780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b69fa-920e-4f96-87dc-a81efd90a520 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:47.492177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 411/850] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:47.497322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91155f0-f247-465a-81bf-36bf4aac52ee None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:47.497730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91155f0-f247-465a-81bf-36bf4aac52ee None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:47.497853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91155f0-f247-465a-81bf-36bf4aac52ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:47.516556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-d91155f0-f247-465a-81bf-36bf4aac52ee None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:47.517720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d91155f0-f247-465a-81bf-36bf4aac52ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:47.583056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 440/851] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:47.854850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b69fa-920e-4f96-87dc-a81efd90a520 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:47.948235 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:47.954661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b69fa-920e-4f96-87dc-a81efd90a520 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:47.987438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6b2e0-2fc2-4f8b-8834-a9b1b903d786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:47.988069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6b2e0-2fc2-4f8b-8834-a9b1b903d786 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:47.988342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6b2e0-2fc2-4f8b-8834-a9b1b903d786 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:47.988606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6b2e0-2fc2-4f8b-8834-a9b1b903d786 None admin] PATH_INFO: `/v3/roles/1cc2ea78f84c4491967839cfead3fd26` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:47.989929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6b2e0-2fc2-4f8b-8834-a9b1b903d786 None admin] RBAC: Authorizing `identity:get_role(role_id=1cc2ea78f84c4491967839cfead3fd26)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:47.991272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6b2e0-2fc2-4f8b-8834-a9b1b903d786 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:47.996586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 412/852] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:43:47 2022] GET /identity/v3/roles/1cc2ea78f84c4491967839cfead3fd26 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:48.002438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91155f0-f247-465a-81bf-36bf4aac52ee None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:48.080269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:48.100937 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91155f0-f247-465a-81bf-36bf4aac52ee None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:48.134463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d7558-2653-437e-a1a3-9b07f20a708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:48.135019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d7558-2653-437e-a1a3-9b07f20a708f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:48.135292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d7558-2653-437e-a1a3-9b07f20a708f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:48.135560 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d7558-2653-437e-a1a3-9b07f20a708f None admin] PATH_INFO: `/v3/users/fb7e99f8b8ae4b15ba9f81a34cae8b32` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:48.136689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d7558-2653-437e-a1a3-9b07f20a708f None admin] RBAC: Authorizing `identity:get_user(user_id=fb7e99f8b8ae4b15ba9f81a34cae8b32)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:48.138119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d7558-2653-437e-a1a3-9b07f20a708f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:48.139669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 441/853] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:43:48 2022] GET /identity/v3/users/fb7e99f8b8ae4b15ba9f81a34cae8b32 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:48.145725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6b2e0-2fc2-4f8b-8834-a9b1b903d786 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:48.241739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6b2e0-2fc2-4f8b-8834-a9b1b903d786 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:48.274589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b3784-4f09-41e8-a80e-4d2851bdf853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:48.275168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b3784-4f09-41e8-a80e-4d2851bdf853 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:48.275439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b3784-4f09-41e8-a80e-4d2851bdf853 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:48.275704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b3784-4f09-41e8-a80e-4d2851bdf853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:48.304822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:48.340112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b3784-4f09-41e8-a80e-4d2851bdf853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:48.341524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b3784-4f09-41e8-a80e-4d2851bdf853 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:48.436001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 413/854] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:43:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:43:48.441922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d7558-2653-437e-a1a3-9b07f20a708f None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:48.540180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d7558-2653-437e-a1a3-9b07f20a708f None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:48.573077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4004e8-c285-4e87-a33f-95cb3187d12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:48.573687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4004e8-c285-4e87-a33f-95cb3187d12e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:48.573962 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4004e8-c285-4e87-a33f-95cb3187d12e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:48.574231 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4004e8-c285-4e87-a33f-95cb3187d12e None admin] PATH_INFO: `/v3/projects/cbe42d45af6a44d0b684f98377f31cbc` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:48.575325 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4004e8-c285-4e87-a33f-95cb3187d12e None admin] RBAC: Authorizing `identity:get_project(project_id=cbe42d45af6a44d0b684f98377f31cbc)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:48.576785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4004e8-c285-4e87-a33f-95cb3187d12e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:48.578399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 442/855] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:43:48 2022] GET /identity/v3/projects/cbe42d45af6a44d0b684f98377f31cbc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:48.584721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b3784-4f09-41e8-a80e-4d2851bdf853 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:48.646048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:48.683243 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b3784-4f09-41e8-a80e-4d2851bdf853 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:48.716701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec4231-a531-49c1-a79d-910c503f898a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:48.717219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec4231-a531-49c1-a79d-910c503f898a None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:48.717489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec4231-a531-49c1-a79d-910c503f898a None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:48.717751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec4231-a531-49c1-a79d-910c503f898a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:48.718453 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec4231-a531-49c1-a79d-910c503f898a None admin] RBAC: Adding query filter params (role.id=1cc2ea78f84c4491967839cfead3fd26, scope.project.id=cbe42d45af6a44d0b684f98377f31cbc, user.id=fb7e99f8b8ae4b15ba9f81a34cae8b32) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:48.718750 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec4231-a531-49c1-a79d-910c503f898a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:48.720170 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec4231-a531-49c1-a79d-910c503f898a None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:48.725590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 414/856] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:43:48 2022] GET /identity/v3/role_assignments?user.id=fb7e99f8b8ae4b15ba9f81a34cae8b32&scope.project.id=cbe42d45af6a44d0b684f98377f31cbc&role.id=1cc2ea78f84c4491967839cfead3fd26 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:50.885433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b8713-dd5c-4327-8115-3f3dec724e60 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:50.886014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b8713-dd5c-4327-8115-3f3dec724e60 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:50.886014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b8713-dd5c-4327-8115-3f3dec724e60 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:50.887281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 443/857] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:50.892946 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35c419-7fe9-473e-9694-b6a88d386ac4 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:50.893379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35c419-7fe9-473e-9694-b6a88d386ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:50.893477 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35c419-7fe9-473e-9694-b6a88d386ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:50.912037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ca35c419-7fe9-473e-9694-b6a88d386ac4 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:50.913186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ca35c419-7fe9-473e-9694-b6a88d386ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:50.975964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 415/858] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:43:51.244201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b8713-dd5c-4327-8115-3f3dec724e60 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:51.259401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:51.343658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b8713-dd5c-4327-8115-3f3dec724e60 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:51.376841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caecb0a-6336-4579-8def-c33175a3af8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:51.377453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caecb0a-6336-4579-8def-c33175a3af8f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:51.377725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caecb0a-6336-4579-8def-c33175a3af8f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:51.377991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caecb0a-6336-4579-8def-c33175a3af8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:51.379074 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caecb0a-6336-4579-8def-c33175a3af8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:51.380540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caecb0a-6336-4579-8def-c33175a3af8f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:51.382083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 444/859] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:51.387892 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35c419-7fe9-473e-9694-b6a88d386ac4 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:51.444526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:51.485458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35c419-7fe9-473e-9694-b6a88d386ac4 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:51.519325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3ba9a-6858-4947-bfbe-30dc7393d872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:51.519850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3ba9a-6858-4947-bfbe-30dc7393d872 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:51.520145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3ba9a-6858-4947-bfbe-30dc7393d872 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:51.520409 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3ba9a-6858-4947-bfbe-30dc7393d872 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:51.521044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3ba9a-6858-4947-bfbe-30dc7393d872 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:51.522438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3ba9a-6858-4947-bfbe-30dc7393d872 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:51.548653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 416/860] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:43:51 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:53.722403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8481aa-5349-4482-9472-ef1b5b17e67e None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:53.722901 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8481aa-5349-4482-9472-ef1b5b17e67e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:53.722901 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8481aa-5349-4482-9472-ef1b5b17e67e None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:53.724229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 445/861] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:53.730011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80b780-8330-43b8-ac5d-d68b52a3de70 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:53.730432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80b780-8330-43b8-ac5d-d68b52a3de70 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:53.730532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80b780-8330-43b8-ac5d-d68b52a3de70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:53.748782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-dc80b780-8330-43b8-ac5d-d68b52a3de70 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:53.749935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-dc80b780-8330-43b8-ac5d-d68b52a3de70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:53.813382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 417/862] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:54.085563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8481aa-5349-4482-9472-ef1b5b17e67e None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:54.166427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:54.185513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8481aa-5349-4482-9472-ef1b5b17e67e None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:54.219131 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ef93b-1cd3-4397-a442-9ea9f29f068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:54.219733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef93b-1cd3-4397-a442-9ea9f29f068f None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:54.220032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef93b-1cd3-4397-a442-9ea9f29f068f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:54.220300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef93b-1cd3-4397-a442-9ea9f29f068f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:54.221370 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ef93b-1cd3-4397-a442-9ea9f29f068f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:54.222783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ef93b-1cd3-4397-a442-9ea9f29f068f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:54.224339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 446/863] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:43:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:54.230448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80b780-8330-43b8-ac5d-d68b52a3de70 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:54.270800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:54.329821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80b780-8330-43b8-ac5d-d68b52a3de70 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:54.363572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf6360-fa1e-4f7c-aafb-30bb34a63b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:54.364193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf6360-fa1e-4f7c-aafb-30bb34a63b60 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:54.364470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf6360-fa1e-4f7c-aafb-30bb34a63b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:54.364739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf6360-fa1e-4f7c-aafb-30bb34a63b60 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:54.365394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf6360-fa1e-4f7c-aafb-30bb34a63b60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:54.366802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf6360-fa1e-4f7c-aafb-30bb34a63b60 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:54.388024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 418/864] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:43:54 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:43:56.574516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7867ab-5f65-4045-8c2c-480dda7af999 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:56.575064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7867ab-5f65-4045-8c2c-480dda7af999 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:56.575064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7867ab-5f65-4045-8c2c-480dda7af999 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:56.576393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 447/865] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:56.581421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e964f-8e75-40e6-a214-192dfa34a55c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:56.581814 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e964f-8e75-40e6-a214-192dfa34a55c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:56.581948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e964f-8e75-40e6-a214-192dfa34a55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:56.600571 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-514e964f-8e75-40e6-a214-192dfa34a55c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:56.602773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-514e964f-8e75-40e6-a214-192dfa34a55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:56.667374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 419/866] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:43:56.941128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7867ab-5f65-4045-8c2c-480dda7af999 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:57.042459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7867ab-5f65-4045-8c2c-480dda7af999 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:57.076864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c17a5d-b5b7-42da-905f-14d0d40aaabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:57.077476 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c17a5d-b5b7-42da-905f-14d0d40aaabc None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:57.077747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c17a5d-b5b7-42da-905f-14d0d40aaabc None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:57.078011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c17a5d-b5b7-42da-905f-14d0d40aaabc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:57.088327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c17a5d-b5b7-42da-905f-14d0d40aaabc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-c1c17a5d-b5b7-42da-905f-14d0d40aaabc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:43:57.093056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:43:57.094206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:43:57.095259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 448/867] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:43:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:43:57.100274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e964f-8e75-40e6-a214-192dfa34a55c None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:57.125883 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:57.198093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e964f-8e75-40e6-a214-192dfa34a55c None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:57.230795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73597c58-19e4-4774-8a94-412a819e90cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:57.231391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73597c58-19e4-4774-8a94-412a819e90cb None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:57.231664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73597c58-19e4-4774-8a94-412a819e90cb None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:57.231956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73597c58-19e4-4774-8a94-412a819e90cb None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:57.232610 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73597c58-19e4-4774-8a94-412a819e90cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:43:57.232905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73597c58-19e4-4774-8a94-412a819e90cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:57.234250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73597c58-19e4-4774-8a94-412a819e90cb None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:57.242098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 420/868] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:43:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:57.248109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c17a5d-b5b7-42da-905f-14d0d40aaabc None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:57.260970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:57.347124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c17a5d-b5b7-42da-905f-14d0d40aaabc None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:57.380705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c5664-1ffc-45c6-9d3a-f456f86818a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:57.381313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c5664-1ffc-45c6-9d3a-f456f86818a0 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:57.381584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c5664-1ffc-45c6-9d3a-f456f86818a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:57.381848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c5664-1ffc-45c6-9d3a-f456f86818a0 None admin] PATH_INFO: `/v3/users/4bd3103933a74028b48c0de926a13283` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:57.394692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c5664-1ffc-45c6-9d3a-f456f86818a0 None admin] RBAC: Authorizing `identity:get_user(user_id=4bd3103933a74028b48c0de926a13283)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:57.396152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c5664-1ffc-45c6-9d3a-f456f86818a0 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:57.397736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 449/869] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:43:57 2022] GET /identity/v3/users/4bd3103933a74028b48c0de926a13283 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:57.404249 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73597c58-19e4-4774-8a94-412a819e90cb None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:57.488329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:57.499999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73597c58-19e4-4774-8a94-412a819e90cb None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:57.532487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e72fc1-019e-408d-8c4b-a62304f0794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:57.533101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e72fc1-019e-408d-8c4b-a62304f0794f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:57.533373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e72fc1-019e-408d-8c4b-a62304f0794f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:57.533641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e72fc1-019e-408d-8c4b-a62304f0794f None admin] PATH_INFO: `/v3/projects/c0c2192216a44a4da3be430bb7ebb8ff` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:57.534729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e72fc1-019e-408d-8c4b-a62304f0794f None admin] RBAC: Authorizing `identity:get_project(project_id=c0c2192216a44a4da3be430bb7ebb8ff)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:57.536179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e72fc1-019e-408d-8c4b-a62304f0794f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:57.537762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 421/870] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:43:57 2022] GET /identity/v3/projects/c0c2192216a44a4da3be430bb7ebb8ff => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:43:57.544340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c5664-1ffc-45c6-9d3a-f456f86818a0 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:43:57.607709 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:43:57.640100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c5664-1ffc-45c6-9d3a-f456f86818a0 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:43:57.672777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76f5d4-f4c6-4c55-aa39-385301b460f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:43:57.673417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76f5d4-f4c6-4c55-aa39-385301b460f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:57.673695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76f5d4-f4c6-4c55-aa39-385301b460f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:57.673958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76f5d4-f4c6-4c55-aa39-385301b460f4 None admin] PATH_INFO: `/v3/projects/c0c2192216a44a4da3be430bb7ebb8ff/users/4bd3103933a74028b48c0de926a13283/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:57.675807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76f5d4-f4c6-4c55-aa39-385301b460f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0c2192216a44a4da3be430bb7ebb8ff, user_id=4bd3103933a74028b48c0de926a13283, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:43:57.677247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76f5d4-f4c6-4c55-aa39-385301b460f4 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:43:57.690382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 450/871] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:43:57 2022] PUT /identity/v3/projects/c0c2192216a44a4da3be430bb7ebb8ff/users/4bd3103933a74028b48c0de926a13283/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:43:59.813784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368708eb-caea-42ed-ace2-9c2912be953a None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:59.814314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368708eb-caea-42ed-ace2-9c2912be953a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:59.814314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368708eb-caea-42ed-ace2-9c2912be953a None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:59.815621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 422/872] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:43:59.820969 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f002fd-c68b-4e6d-8d5a-5ee35029c343 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:43:59.821483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f002fd-c68b-4e6d-8d5a-5ee35029c343 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:43:59.821586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f002fd-c68b-4e6d-8d5a-5ee35029c343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:43:59.840307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e8f002fd-c68b-4e6d-8d5a-5ee35029c343 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:43:59.841485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e8f002fd-c68b-4e6d-8d5a-5ee35029c343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:43:59.906687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 451/873] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:43:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:44:00.182476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368708eb-caea-42ed-ace2-9c2912be953a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:00.283132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368708eb-caea-42ed-ace2-9c2912be953a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:00.316409 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4cbb4-bbd1-4058-be7c-8501dd9e3869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:00.317019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4cbb4-bbd1-4058-be7c-8501dd9e3869 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:00.317293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4cbb4-bbd1-4058-be7c-8501dd9e3869 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:00.317562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4cbb4-bbd1-4058-be7c-8501dd9e3869 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:00.327978 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4cbb4-bbd1-4058-be7c-8501dd9e3869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-1cd4cbb4-bbd1-4058-be7c-8501dd9e3869 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:44:00.332692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:44:00.333789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:44:00.334853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 423/874] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:44:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:44:00.340178 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f002fd-c68b-4e6d-8d5a-5ee35029c343 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:00.397066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:00.438030 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f002fd-c68b-4e6d-8d5a-5ee35029c343 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:00.470657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a70b4-b40a-4e8e-86c7-559147631558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:00.471248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a70b4-b40a-4e8e-86c7-559147631558 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:00.471522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a70b4-b40a-4e8e-86c7-559147631558 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:00.471789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a70b4-b40a-4e8e-86c7-559147631558 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:00.472470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a70b4-b40a-4e8e-86c7-559147631558 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:44:00.472766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a70b4-b40a-4e8e-86c7-559147631558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:00.474130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a70b4-b40a-4e8e-86c7-559147631558 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:00.481745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 452/875] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:44:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:44:00.487630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4cbb4-bbd1-4058-be7c-8501dd9e3869 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:00.506168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:00.584579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4cbb4-bbd1-4058-be7c-8501dd9e3869 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:00.617661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c59da-b3ca-4f5a-a1ef-955afb771f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:00.618684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c59da-b3ca-4f5a-a1ef-955afb771f28 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:00.618953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c59da-b3ca-4f5a-a1ef-955afb771f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:00.619218 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c59da-b3ca-4f5a-a1ef-955afb771f28 None admin] PATH_INFO: `/v3/users/4bd3103933a74028b48c0de926a13283` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:00.620373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c59da-b3ca-4f5a-a1ef-955afb771f28 None admin] RBAC: Authorizing `identity:get_user(user_id=4bd3103933a74028b48c0de926a13283)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:00.621776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c59da-b3ca-4f5a-a1ef-955afb771f28 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:00.623310 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 424/876] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:44:00 2022] GET /identity/v3/users/4bd3103933a74028b48c0de926a13283 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:44:00.629070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a70b4-b40a-4e8e-86c7-559147631558 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:00.727727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a70b4-b40a-4e8e-86c7-559147631558 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:00.746798 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:00.760832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3026a-c5cf-49bb-82cb-fee9ced118ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:00.761422 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3026a-c5cf-49bb-82cb-fee9ced118ed None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:00.761698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3026a-c5cf-49bb-82cb-fee9ced118ed None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:00.761991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3026a-c5cf-49bb-82cb-fee9ced118ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:00.827044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3026a-c5cf-49bb-82cb-fee9ced118ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:00.828519 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3026a-c5cf-49bb-82cb-fee9ced118ed None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:00.922440 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 453/877] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:44:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:44:00.928328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c59da-b3ca-4f5a-a1ef-955afb771f28 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:01.009462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:01.025793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c59da-b3ca-4f5a-a1ef-955afb771f28 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:01.057702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8f097-db02-4cf8-b374-9a1cc45c76e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:01.058319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8f097-db02-4cf8-b374-9a1cc45c76e1 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:01.058592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8f097-db02-4cf8-b374-9a1cc45c76e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:01.058858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8f097-db02-4cf8-b374-9a1cc45c76e1 None admin] PATH_INFO: `/v3/projects/c0c2192216a44a4da3be430bb7ebb8ff` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:01.059977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8f097-db02-4cf8-b374-9a1cc45c76e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c0c2192216a44a4da3be430bb7ebb8ff)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:01.061391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8f097-db02-4cf8-b374-9a1cc45c76e1 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:01.062987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 425/878] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:44:00 2022] GET /identity/v3/projects/c0c2192216a44a4da3be430bb7ebb8ff => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:44:01.069117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3026a-c5cf-49bb-82cb-fee9ced118ed None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:01.090219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:01.167630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3026a-c5cf-49bb-82cb-fee9ced118ed None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:01.200760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621e7f9-52b2-4770-8027-919d013457c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:01.201344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621e7f9-52b2-4770-8027-919d013457c8 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:01.201619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621e7f9-52b2-4770-8027-919d013457c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:01.201891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621e7f9-52b2-4770-8027-919d013457c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:01.202605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621e7f9-52b2-4770-8027-919d013457c8 None admin] RBAC: Adding query filter params (role.id=45b67ba9922c4b16baae3930d3de1b6c, scope.project.id=c0c2192216a44a4da3be430bb7ebb8ff, user.id=4bd3103933a74028b48c0de926a13283) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:44:01.202906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621e7f9-52b2-4770-8027-919d013457c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:01.204351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621e7f9-52b2-4770-8027-919d013457c8 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:01.209868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 454/879] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:44:01 2022] GET /identity/v3/role_assignments?user.id=4bd3103933a74028b48c0de926a13283&scope.project.id=c0c2192216a44a4da3be430bb7ebb8ff&role.id=45b67ba9922c4b16baae3930d3de1b6c => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:44:03.389331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694193fd-3e4a-439b-9923-441d2e6b0814 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:03.389877 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694193fd-3e4a-439b-9923-441d2e6b0814 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:03.389877 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694193fd-3e4a-439b-9923-441d2e6b0814 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:03.391187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 426/880] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:44:03.396731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62537753-dd6f-4d7c-8a3a-e07016efacbc None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:03.397159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62537753-dd6f-4d7c-8a3a-e07016efacbc None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:03.397259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62537753-dd6f-4d7c-8a3a-e07016efacbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:03.416128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-62537753-dd6f-4d7c-8a3a-e07016efacbc None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:44:03.417294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-62537753-dd6f-4d7c-8a3a-e07016efacbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:44:03.481969 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 455/881] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:44:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:44:03.752755 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694193fd-3e4a-439b-9923-441d2e6b0814 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:03.850648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694193fd-3e4a-439b-9923-441d2e6b0814 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:03.884602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcbde0-23c3-4c63-ab6a-27018b9809f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:03.885178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcbde0-23c3-4c63-ab6a-27018b9809f1 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:03.885447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcbde0-23c3-4c63-ab6a-27018b9809f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:03.885710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcbde0-23c3-4c63-ab6a-27018b9809f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:03.886356 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcbde0-23c3-4c63-ab6a-27018b9809f1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:03.887686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcbde0-23c3-4c63-ab6a-27018b9809f1 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:03.903853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 427/882] 10.222.0.27 () {60 vars in 1138 bytes} [Fri Oct 14 17:44:03 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:44:06.099011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5b00c-5679-4b8a-8fd0-ccb19e0bb72a None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:06.099576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5b00c-5679-4b8a-8fd0-ccb19e0bb72a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:06.099576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5b00c-5679-4b8a-8fd0-ccb19e0bb72a None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:06.101014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 456/883] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:44:06.106631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccdb0b-aa08-4d5e-8345-92215a24a1aa None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:06.107074 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccdb0b-aa08-4d5e-8345-92215a24a1aa None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:06.107176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccdb0b-aa08-4d5e-8345-92215a24a1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:06.124930 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-8bccdb0b-aa08-4d5e-8345-92215a24a1aa None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:44:06.126088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8bccdb0b-aa08-4d5e-8345-92215a24a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:44:06.133144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:06.192340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 428/884] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:44:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:44:06.465711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5b00c-5679-4b8a-8fd0-ccb19e0bb72a None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:06.481131 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:06.564988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5b00c-5679-4b8a-8fd0-ccb19e0bb72a None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:06.597326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1b9fa-e942-4166-a8b5-38d935c91b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:06.597931 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1b9fa-e942-4166-a8b5-38d935c91b79 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:06.598202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1b9fa-e942-4166-a8b5-38d935c91b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:06.598468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1b9fa-e942-4166-a8b5-38d935c91b79 None admin] PATH_INFO: `/v3/domains/2ead2fe1f01943c885463b2a30cf1777` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:06.613312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b1b9fa-e942-4166-a8b5-38d935c91b79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2ead2fe1f01943c885463b2a30cf1777)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:06.613312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b1b9fa-e942-4166-a8b5-38d935c91b79 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:06.613312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 457/885] 10.222.0.27 () {56 vars in 1183 bytes} [Fri Oct 14 17:44:06 2022] GET /identity/v3/domains/2ead2fe1f01943c885463b2a30cf1777 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:44:06.618067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccdb0b-aa08-4d5e-8345-92215a24a1aa None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:06.718546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccdb0b-aa08-4d5e-8345-92215a24a1aa None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:06.751643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192db5cd-7837-4a3c-9893-21fd9386dae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:06.752279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db5cd-7837-4a3c-9893-21fd9386dae2 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:06.752557 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db5cd-7837-4a3c-9893-21fd9386dae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:06.752825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db5cd-7837-4a3c-9893-21fd9386dae2 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:06.753471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192db5cd-7837-4a3c-9893-21fd9386dae2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:06.754882 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192db5cd-7837-4a3c-9893-21fd9386dae2 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:06.768883 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:06.781122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 429/886] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:44:06 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:44:08.955621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4a2a8-a172-4d41-b262-14aa9faeabf9 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:08.956317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4a2a8-a172-4d41-b262-14aa9faeabf9 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:08.956317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4a2a8-a172-4d41-b262-14aa9faeabf9 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:08.957491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 458/887] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:44:08.962649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910e7a4-3a0d-4eae-b980-50b88e5a133e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:08.963052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910e7a4-3a0d-4eae-b980-50b88e5a133e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:08.963175 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910e7a4-3a0d-4eae-b980-50b88e5a133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:08.981650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-4910e7a4-3a0d-4eae-b980-50b88e5a133e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:44:08.982800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-4910e7a4-3a0d-4eae-b980-50b88e5a133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:44:09.046660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 430/888] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:44:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:44:09.317915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4a2a8-a172-4d41-b262-14aa9faeabf9 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:09.392571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:09.418113 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4a2a8-a172-4d41-b262-14aa9faeabf9 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:09.450266 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc07d1-e922-4429-8dfa-da63dd89ff54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:09.450874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc07d1-e922-4429-8dfa-da63dd89ff54 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:09.451144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc07d1-e922-4429-8dfa-da63dd89ff54 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:09.451405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc07d1-e922-4429-8dfa-da63dd89ff54 None admin] PATH_INFO: `/v3/domains/2ead2fe1f01943c885463b2a30cf1777` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:09.452494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc07d1-e922-4429-8dfa-da63dd89ff54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2ead2fe1f01943c885463b2a30cf1777)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:09.453913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc07d1-e922-4429-8dfa-da63dd89ff54 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:09.455444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 459/889] 10.222.0.27 () {56 vars in 1183 bytes} [Fri Oct 14 17:44:09 2022] GET /identity/v3/domains/2ead2fe1f01943c885463b2a30cf1777 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:44:09.461183 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910e7a4-3a0d-4eae-b980-50b88e5a133e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:09.554676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910e7a4-3a0d-4eae-b980-50b88e5a133e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:09.577651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:09.589190 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e172593-51e9-4e14-9418-e62ee224cbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:09.589780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e172593-51e9-4e14-9418-e62ee224cbbc None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:09.590049 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e172593-51e9-4e14-9418-e62ee224cbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:09.590314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e172593-51e9-4e14-9418-e62ee224cbbc None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:09.590944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e172593-51e9-4e14-9418-e62ee224cbbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:09.592374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e172593-51e9-4e14-9418-e62ee224cbbc None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:09.612053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 431/890] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:44:09 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:44:11.778995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ef16-78b1-4c74-996b-a740d33daf6f None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:11.779511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ef16-78b1-4c74-996b-a740d33daf6f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:11.779511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ef16-78b1-4c74-996b-a740d33daf6f None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:11.780810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 460/891] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:44:11.786030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0ecc9-1f36-4d71-8f73-85a0dc36a5cd None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:11.786409 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0ecc9-1f36-4d71-8f73-85a0dc36a5cd None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:11.786568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0ecc9-1f36-4d71-8f73-85a0dc36a5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:11.804468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-62d0ecc9-1f36-4d71-8f73-85a0dc36a5cd None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:44:11.806636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-62d0ecc9-1f36-4d71-8f73-85a0dc36a5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:44:11.870693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 432/892] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:44:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:44:12.143158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ef16-78b1-4c74-996b-a740d33daf6f None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:12.241184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ef16-78b1-4c74-996b-a740d33daf6f None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:12.273644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1e0ee-20d1-4b71-bdac-706aaef0cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:12.274794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1e0ee-20d1-4b71-bdac-706aaef0cb48 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:12.275067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1e0ee-20d1-4b71-bdac-706aaef0cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:12.275328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1e0ee-20d1-4b71-bdac-706aaef0cb48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:12.285484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1e0ee-20d1-4b71-bdac-706aaef0cb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-30a1e0ee-20d1-4b71-bdac-706aaef0cb48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:44:12.290896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:44:12.292060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:44:12.293098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 461/893] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:44:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:44:12.297900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0ecc9-1f36-4d71-8f73-85a0dc36a5cd None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:12.393457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0ecc9-1f36-4d71-8f73-85a0dc36a5cd None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:12.404950 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:12.429631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e6b68-5ddc-4882-abb7-c5fddba3793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:12.430320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e6b68-5ddc-4882-abb7-c5fddba3793f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:12.430753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e6b68-5ddc-4882-abb7-c5fddba3793f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:12.431025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e6b68-5ddc-4882-abb7-c5fddba3793f None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:12.431673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e6b68-5ddc-4882-abb7-c5fddba3793f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:44:12.432000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e6b68-5ddc-4882-abb7-c5fddba3793f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:12.433348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e6b68-5ddc-4882-abb7-c5fddba3793f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:12.440972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 433/894] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:44:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:44:12.446945 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1e0ee-20d1-4b71-bdac-706aaef0cb48 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:12.449803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:12.544298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1e0ee-20d1-4b71-bdac-706aaef0cb48 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:12.576424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba6c20-bbe3-4905-97bd-557bdb458d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:12.577023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba6c20-bbe3-4905-97bd-557bdb458d89 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:12.577295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba6c20-bbe3-4905-97bd-557bdb458d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:12.577560 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba6c20-bbe3-4905-97bd-557bdb458d89 None admin] PATH_INFO: `/v3/users/eed462df027e4ffbacef152654db17d1` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:12.590588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba6c20-bbe3-4905-97bd-557bdb458d89 None admin] RBAC: Authorizing `identity:get_user(user_id=eed462df027e4ffbacef152654db17d1)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:12.592046 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba6c20-bbe3-4905-97bd-557bdb458d89 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:12.593611 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 462/895] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:44:12 2022] GET /identity/v3/users/eed462df027e4ffbacef152654db17d1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:44:12.599323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e6b68-5ddc-4882-abb7-c5fddba3793f None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:12.698396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e6b68-5ddc-4882-abb7-c5fddba3793f None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:12.731224 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646720b-9f7c-4cc2-b110-ffb0156b18e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:12.731830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646720b-9f7c-4cc2-b110-ffb0156b18e7 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:12.732130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646720b-9f7c-4cc2-b110-ffb0156b18e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:12.732394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646720b-9f7c-4cc2-b110-ffb0156b18e7 None admin] PATH_INFO: `/v3/projects/41a250948c3a4f7fb7164a4b3dc818d4` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:12.733478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646720b-9f7c-4cc2-b110-ffb0156b18e7 None admin] RBAC: Authorizing `identity:get_project(project_id=41a250948c3a4f7fb7164a4b3dc818d4)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:12.734878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646720b-9f7c-4cc2-b110-ffb0156b18e7 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:12.736492 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 434/896] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:44:12 2022] GET /identity/v3/projects/41a250948c3a4f7fb7164a4b3dc818d4 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:44:12.742439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba6c20-bbe3-4905-97bd-557bdb458d89 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:12.800398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:12.840118 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba6c20-bbe3-4905-97bd-557bdb458d89 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:12.873509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5babe7-f78b-40c9-8cc9-75ac855c5387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:12.874141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5babe7-f78b-40c9-8cc9-75ac855c5387 None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:12.874414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5babe7-f78b-40c9-8cc9-75ac855c5387 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:12.874676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5babe7-f78b-40c9-8cc9-75ac855c5387 None admin] PATH_INFO: `/v3/projects/41a250948c3a4f7fb7164a4b3dc818d4/users/eed462df027e4ffbacef152654db17d1/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:12.876533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5babe7-f78b-40c9-8cc9-75ac855c5387 None admin] RBAC: Authorizing `identity:create_grant(project_id=41a250948c3a4f7fb7164a4b3dc818d4, user_id=eed462df027e4ffbacef152654db17d1, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:12.877943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5babe7-f78b-40c9-8cc9-75ac855c5387 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:12.890701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 463/897] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:44:12 2022] PUT /identity/v3/projects/41a250948c3a4f7fb7164a4b3dc818d4/users/eed462df027e4ffbacef152654db17d1/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:44:15.025681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b71868-8034-46e6-ab05-29e5b9295c79 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:15.026247 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b71868-8034-46e6-ab05-29e5b9295c79 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:15.026247 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b71868-8034-46e6-ab05-29e5b9295c79 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:15.027534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 435/898] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:44:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:44:15.032992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c362fbb-dc84-4b90-8e58-3151323c34cb None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:15.033371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c362fbb-dc84-4b90-8e58-3151323c34cb None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:15.033525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c362fbb-dc84-4b90-8e58-3151323c34cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:15.052299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8c362fbb-dc84-4b90-8e58-3151323c34cb None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:44:15.053485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8c362fbb-dc84-4b90-8e58-3151323c34cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:44:15.118603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 464/899] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:44:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:44:15.384392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b71868-8034-46e6-ab05-29e5b9295c79 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:15.428150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:15.485640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b71868-8034-46e6-ab05-29e5b9295c79 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:15.518015 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cb805-3c8d-4812-8d6f-bc1746d2aae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:15.518622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cb805-3c8d-4812-8d6f-bc1746d2aae6 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:15.518891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cb805-3c8d-4812-8d6f-bc1746d2aae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:15.519152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cb805-3c8d-4812-8d6f-bc1746d2aae6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:15.529634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cb805-3c8d-4812-8d6f-bc1746d2aae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-363cb805-3c8d-4812-8d6f-bc1746d2aae6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:44:15.534289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:44:15.535373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:44:15.536374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 436/900] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:44:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:44:15.542036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c362fbb-dc84-4b90-8e58-3151323c34cb None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:15.589468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:15.638501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c362fbb-dc84-4b90-8e58-3151323c34cb None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:15.671590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6bab4-eb3a-4a22-8113-7c6f268eed83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:15.672204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6bab4-eb3a-4a22-8113-7c6f268eed83 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:15.672482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6bab4-eb3a-4a22-8113-7c6f268eed83 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:15.672752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6bab4-eb3a-4a22-8113-7c6f268eed83 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:15.673406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6bab4-eb3a-4a22-8113-7c6f268eed83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:44:15.673703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6bab4-eb3a-4a22-8113-7c6f268eed83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:15.675063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6bab4-eb3a-4a22-8113-7c6f268eed83 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:15.682851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 465/901] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:44:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:44:15.688706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cb805-3c8d-4812-8d6f-bc1746d2aae6 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:15.783585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:15.786264 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cb805-3c8d-4812-8d6f-bc1746d2aae6 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:15.819530 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89cd8e-4e22-46f5-a303-a4bf3b98e784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:15.820178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89cd8e-4e22-46f5-a303-a4bf3b98e784 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:15.820456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89cd8e-4e22-46f5-a303-a4bf3b98e784 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:15.821245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89cd8e-4e22-46f5-a303-a4bf3b98e784 None admin] PATH_INFO: `/v3/users/eed462df027e4ffbacef152654db17d1` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:15.822475 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89cd8e-4e22-46f5-a303-a4bf3b98e784 None admin] RBAC: Authorizing `identity:get_user(user_id=eed462df027e4ffbacef152654db17d1)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:15.823913 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89cd8e-4e22-46f5-a303-a4bf3b98e784 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:15.825466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 437/902] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Oct 14 17:44:15 2022] GET /identity/v3/users/eed462df027e4ffbacef152654db17d1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:44:15.831152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6bab4-eb3a-4a22-8113-7c6f268eed83 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:15.927134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6bab4-eb3a-4a22-8113-7c6f268eed83 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:15.939516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:15.961211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda19011-b666-4182-a8ee-98f605431aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:15.962630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda19011-b666-4182-a8ee-98f605431aee None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:15.962908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda19011-b666-4182-a8ee-98f605431aee None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:15.963175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda19011-b666-4182-a8ee-98f605431aee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:16.028066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda19011-b666-4182-a8ee-98f605431aee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:16.029503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda19011-b666-4182-a8ee-98f605431aee None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:16.123423 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 466/903] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:44:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:44:16.129315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89cd8e-4e22-46f5-a303-a4bf3b98e784 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:16.225417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89cd8e-4e22-46f5-a303-a4bf3b98e784 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:16.257212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31b631-b6fd-4d06-bec0-2227819d876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:16.257820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31b631-b6fd-4d06-bec0-2227819d876b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:16.258093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31b631-b6fd-4d06-bec0-2227819d876b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:16.259000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31b631-b6fd-4d06-bec0-2227819d876b None admin] PATH_INFO: `/v3/projects/41a250948c3a4f7fb7164a4b3dc818d4` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:16.260130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31b631-b6fd-4d06-bec0-2227819d876b None admin] RBAC: Authorizing `identity:get_project(project_id=41a250948c3a4f7fb7164a4b3dc818d4)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:16.261540 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31b631-b6fd-4d06-bec0-2227819d876b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:16.263111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 438/904] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Oct 14 17:44:16 2022] GET /identity/v3/projects/41a250948c3a4f7fb7164a4b3dc818d4 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:44:16.269316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda19011-b666-4182-a8ee-98f605431aee None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:44:16.282872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:16.366052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda19011-b666-4182-a8ee-98f605431aee None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:44:16.400025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702273b-b188-4b7d-8abe-7833025fecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:44:16.400614 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702273b-b188-4b7d-8abe-7833025fecb3 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:44:16.400887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702273b-b188-4b7d-8abe-7833025fecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:44:16.401152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702273b-b188-4b7d-8abe-7833025fecb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:44:16.401854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702273b-b188-4b7d-8abe-7833025fecb3 None admin] RBAC: Adding query filter params (role.id=45b67ba9922c4b16baae3930d3de1b6c, scope.project.id=41a250948c3a4f7fb7164a4b3dc818d4, user.id=eed462df027e4ffbacef152654db17d1) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:44:16.402151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702273b-b188-4b7d-8abe-7833025fecb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:44:16.403559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702273b-b188-4b7d-8abe-7833025fecb3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:44:16.409177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 467/905] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:44:16 2022] GET /identity/v3/role_assignments?user.id=eed462df027e4ffbacef152654db17d1&scope.project.id=41a250948c3a4f7fb7164a4b3dc818d4&role.id=45b67ba9922c4b16baae3930d3de1b6c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:44:26.253141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=82 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:44:26.406584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=62 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:12.294103 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab535b-7d22-44c4-a53f-326335b03b98 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:12.294727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab535b-7d22-44c4-a53f-326335b03b98 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:12.294992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab535b-7d22-44c4-a53f-326335b03b98 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:12.296917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 439/906] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:45:12.303348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb8bfb-4d39-4945-a3a5-059b32c3b59b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:12.303924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb8bfb-4d39-4945-a3a5-059b32c3b59b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:12.304037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb8bfb-4d39-4945-a3a5-059b32c3b59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:12.328833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-22fb8bfb-4d39-4945-a3a5-059b32c3b59b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:45:12.330028 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-22fb8bfb-4d39-4945-a3a5-059b32c3b59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:45:12.427776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 468/907] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:45:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:45:12.713466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab535b-7d22-44c4-a53f-326335b03b98 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:12.869687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab535b-7d22-44c4-a53f-326335b03b98 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:12.924628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a857f6d-d543-4404-91f5-2d47e38df82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:12.925273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a857f6d-d543-4404-91f5-2d47e38df82f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:12.925554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a857f6d-d543-4404-91f5-2d47e38df82f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:12.925823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a857f6d-d543-4404-91f5-2d47e38df82f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:12.937309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a857f6d-d543-4404-91f5-2d47e38df82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:12.938901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a857f6d-d543-4404-91f5-2d47e38df82f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-9a857f6d-d543-4404-91f5-2d47e38df82f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:45:12.956992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:12.958189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:45:12.959191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:45:12.959191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:12.959191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:12.959191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 440/908] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:12.964971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb8bfb-4d39-4945-a3a5-059b32c3b59b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:13.085957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb8bfb-4d39-4945-a3a5-059b32c3b59b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:13.129286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:13.137636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc2f11-3b76-4d14-a78c-6aaea1ab34c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:13.138375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc2f11-3b76-4d14-a78c-6aaea1ab34c3 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:13.138656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc2f11-3b76-4d14-a78c-6aaea1ab34c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:13.139080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc2f11-3b76-4d14-a78c-6aaea1ab34c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:13.139857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc2f11-3b76-4d14-a78c-6aaea1ab34c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:13.140323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc2f11-3b76-4d14-a78c-6aaea1ab34c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:13.144643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc2f11-3b76-4d14-a78c-6aaea1ab34c3 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:13.161075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 469/909] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:45:13.168374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a857f6d-d543-4404-91f5-2d47e38df82f None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:13.237705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:13.308475 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a857f6d-d543-4404-91f5-2d47e38df82f None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:13.351898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb4c94-9b80-4c3f-8bd2-b02586f937f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:13.356016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb4c94-9b80-4c3f-8bd2-b02586f937f2 None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:13.356505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb4c94-9b80-4c3f-8bd2-b02586f937f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:13.356937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb4c94-9b80-4c3f-8bd2-b02586f937f2 None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:13.357792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb4c94-9b80-4c3f-8bd2-b02586f937f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:13.359225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb4c94-9b80-4c3f-8bd2-b02586f937f2 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:13.391936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 441/910] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Oct 14 17:45:13 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:45:15.671560 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717584dc-5515-47fd-955d-faa3e0ffbdad None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:15.672101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717584dc-5515-47fd-955d-faa3e0ffbdad None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:15.672101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717584dc-5515-47fd-955d-faa3e0ffbdad None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:15.673368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 470/911] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:45:15.679575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0e627-8474-4d65-86d1-bd00f41bd194 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:15.680078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0e627-8474-4d65-86d1-bd00f41bd194 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:15.680305 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0e627-8474-4d65-86d1-bd00f41bd194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:15.702902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-eca0e627-8474-4d65-86d1-bd00f41bd194 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:45:15.704127 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-eca0e627-8474-4d65-86d1-bd00f41bd194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:45:15.783928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 442/912] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:45:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:45:16.053424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717584dc-5515-47fd-955d-faa3e0ffbdad None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:16.180767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717584dc-5515-47fd-955d-faa3e0ffbdad None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:16.221341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675e988-aabf-43bb-929d-ec0c4b938742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:16.222224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675e988-aabf-43bb-929d-ec0c4b938742 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:16.222689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675e988-aabf-43bb-929d-ec0c4b938742 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:16.222961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675e988-aabf-43bb-929d-ec0c4b938742 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:16.235358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675e988-aabf-43bb-929d-ec0c4b938742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-3675e988-aabf-43bb-929d-ec0c4b938742 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:45:16.240574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:16.241552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:16.246156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 471/913] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:45:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:16.248604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0e627-8474-4d65-86d1-bd00f41bd194 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:16.293869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:16.371153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0e627-8474-4d65-86d1-bd00f41bd194 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:16.420200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5074d-658b-45c1-a513-e6bc2896cdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:16.420818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5074d-658b-45c1-a513-e6bc2896cdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:16.421171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5074d-658b-45c1-a513-e6bc2896cdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:16.421437 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5074d-658b-45c1-a513-e6bc2896cdd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:16.422275 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5074d-658b-45c1-a513-e6bc2896cdd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:16.422611 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5074d-658b-45c1-a513-e6bc2896cdd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:16.424319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5074d-658b-45c1-a513-e6bc2896cdd2 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:16.434185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 443/914] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:45:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:16.440664 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675e988-aabf-43bb-929d-ec0c4b938742 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:16.507643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:16.580399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675e988-aabf-43bb-929d-ec0c4b938742 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:16.632200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad403f4-4acb-4734-a6a5-d5552c436a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:16.633157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad403f4-4acb-4734-a6a5-d5552c436a29 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:16.633627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad403f4-4acb-4734-a6a5-d5552c436a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:16.634066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad403f4-4acb-4734-a6a5-d5552c436a29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:16.654438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad403f4-4acb-4734-a6a5-d5552c436a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:16.655925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad403f4-4acb-4734-a6a5-d5552c436a29 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-6ad403f4-4acb-4734-a6a5-d5552c436a29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:45:16.673484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:16.675010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:45:16.676130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:45:16.676130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:16.676130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:16.680872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 472/915] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:16.681014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5074d-658b-45c1-a513-e6bc2896cdd2 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:16.799039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:16.802799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5074d-658b-45c1-a513-e6bc2896cdd2 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:16.845315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57686b51-a236-4ed1-a3a4-fe9aefc96e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:16.845988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57686b51-a236-4ed1-a3a4-fe9aefc96e1b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:16.846297 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57686b51-a236-4ed1-a3a4-fe9aefc96e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:16.846632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57686b51-a236-4ed1-a3a4-fe9aefc96e1b None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:16.847359 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57686b51-a236-4ed1-a3a4-fe9aefc96e1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:16.847686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57686b51-a236-4ed1-a3a4-fe9aefc96e1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:16.849304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57686b51-a236-4ed1-a3a4-fe9aefc96e1b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:16.871670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 444/916] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:16.873601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad403f4-4acb-4734-a6a5-d5552c436a29 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:17.000118 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad403f4-4acb-4734-a6a5-d5552c436a29 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:17.002018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:17.042912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ca1b7-ebf0-48bd-ba15-211f6ee3022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:17.043849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ca1b7-ebf0-48bd-ba15-211f6ee3022e None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:17.044412 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ca1b7-ebf0-48bd-ba15-211f6ee3022e None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:17.044686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ca1b7-ebf0-48bd-ba15-211f6ee3022e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:17.058133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ca1b7-ebf0-48bd-ba15-211f6ee3022e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:17.059824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ca1b7-ebf0-48bd-ba15-211f6ee3022e None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-547ca1b7-ebf0-48bd-ba15-211f6ee3022e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:45:17.073926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:17.075280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:17.077529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:17.077529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:17.077529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:45:17.077529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:45:17.077529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:45:17.077529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:45:17.077529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:45:17.077529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:45:17.077529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 17:45:17.077529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:17.080235 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 473/917] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:17.081437 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57686b51-a236-4ed1-a3a4-fe9aefc96e1b None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:17.196621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57686b51-a236-4ed1-a3a4-fe9aefc96e1b None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:17.239620 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6a9c5-c095-4294-bb18-ef5464c78bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:17.240286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6a9c5-c095-4294-bb18-ef5464c78bbb None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:17.240561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6a9c5-c095-4294-bb18-ef5464c78bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:17.240853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6a9c5-c095-4294-bb18-ef5464c78bbb None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:17.241562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6a9c5-c095-4294-bb18-ef5464c78bbb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:17.241860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6a9c5-c095-4294-bb18-ef5464c78bbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:17.243325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6a9c5-c095-4294-bb18-ef5464c78bbb None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:17.272034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 445/918] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Oct 14 17:45:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:17.273706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ca1b7-ebf0-48bd-ba15-211f6ee3022e None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:17.396539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ca1b7-ebf0-48bd-ba15-211f6ee3022e None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:17.435448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167c5b0-d5f1-4a21-91a5-315700b6e267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:17.436075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167c5b0-d5f1-4a21-91a5-315700b6e267 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:17.436335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167c5b0-d5f1-4a21-91a5-315700b6e267 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:17.436594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167c5b0-d5f1-4a21-91a5-315700b6e267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:17.450478 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167c5b0-d5f1-4a21-91a5-315700b6e267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:17.452134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167c5b0-d5f1-4a21-91a5-315700b6e267 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-c167c5b0-d5f1-4a21-91a5-315700b6e267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:45:17.467600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:17.468673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:45:17.469574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:45:17.469574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:17.469574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:17.469574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 474/919] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:17.476202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6a9c5-c095-4294-bb18-ef5464c78bbb None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:17.498627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:17.593986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6a9c5-c095-4294-bb18-ef5464c78bbb None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:17.630293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee27c9d-2a7c-4f30-8d84-10b7aaf905c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:17.630899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee27c9d-2a7c-4f30-8d84-10b7aaf905c8 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:17.631174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee27c9d-2a7c-4f30-8d84-10b7aaf905c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:17.631443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee27c9d-2a7c-4f30-8d84-10b7aaf905c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:17.632097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee27c9d-2a7c-4f30-8d84-10b7aaf905c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:17.632394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee27c9d-2a7c-4f30-8d84-10b7aaf905c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:17.633751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee27c9d-2a7c-4f30-8d84-10b7aaf905c8 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:17.646101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 446/920] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:17.652347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167c5b0-d5f1-4a21-91a5-315700b6e267 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:17.669660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:17.765797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167c5b0-d5f1-4a21-91a5-315700b6e267 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:17.810425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82160-1351-49d6-855f-11ea5747d871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:17.811052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82160-1351-49d6-855f-11ea5747d871 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:17.811375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82160-1351-49d6-855f-11ea5747d871 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:17.811651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82160-1351-49d6-855f-11ea5747d871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:17.818338 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af82160-1351-49d6-855f-11ea5747d871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:17.819784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af82160-1351-49d6-855f-11ea5747d871 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-1af82160-1351-49d6-855f-11ea5747d871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:45:17.831761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:17.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:17.833740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:45:17.833740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:45:17.833740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:45:17.833740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:45:17.833740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:45:17.833740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:45:17.833740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:45:17.833740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:17.837825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 475/921] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:45:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:17.839371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee27c9d-2a7c-4f30-8d84-10b7aaf905c8 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:17.930631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:17.955359 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee27c9d-2a7c-4f30-8d84-10b7aaf905c8 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:18.002151 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e5e75-ce18-4851-9058-9f5c36bba475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:18.002743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e5e75-ce18-4851-9058-9f5c36bba475 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:18.004161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e5e75-ce18-4851-9058-9f5c36bba475 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:18.004432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e5e75-ce18-4851-9058-9f5c36bba475 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:18.005123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e5e75-ce18-4851-9058-9f5c36bba475 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:18.005422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e5e75-ce18-4851-9058-9f5c36bba475 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:18.006848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e5e75-ce18-4851-9058-9f5c36bba475 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:18.020206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 447/922] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:45:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:45:18.027601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82160-1351-49d6-855f-11ea5747d871 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:18.130806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:18.162352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82160-1351-49d6-855f-11ea5747d871 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:18.197496 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8445d-ca89-4dd0-b6d6-72300966264f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:18.198763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8445d-ca89-4dd0-b6d6-72300966264f None admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:18.199130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8445d-ca89-4dd0-b6d6-72300966264f None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:18.199402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8445d-ca89-4dd0-b6d6-72300966264f None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/73089bc9f85a405b86c4cd3f8cfa830a/roles/e1a3bea83ac349acb6d65b6afb5441d6` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:18.218281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8445d-ca89-4dd0-b6d6-72300966264f None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=73089bc9f85a405b86c4cd3f8cfa830a, role_id=e1a3bea83ac349acb6d65b6afb5441d6)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:18.220080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8445d-ca89-4dd0-b6d6-72300966264f None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:18.234980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 476/923] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:45:18 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/73089bc9f85a405b86c4cd3f8cfa830a/roles/e1a3bea83ac349acb6d65b6afb5441d6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:45:20.402185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec6377-9380-4c09-8394-11f64ac03739 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:20.402817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec6377-9380-4c09-8394-11f64ac03739 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:20.402817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec6377-9380-4c09-8394-11f64ac03739 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:20.404337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 448/924] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:45:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:45:20.410112 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648ab75-c357-413f-8b63-f63f29e41bca None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:20.410572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648ab75-c357-413f-8b63-f63f29e41bca None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:20.410700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648ab75-c357-413f-8b63-f63f29e41bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:20.430895 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0648ab75-c357-413f-8b63-f63f29e41bca None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:45:20.432681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0648ab75-c357-413f-8b63-f63f29e41bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:45:20.502578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 477/925] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:45:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:45:20.772780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec6377-9380-4c09-8394-11f64ac03739 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:20.881561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec6377-9380-4c09-8394-11f64ac03739 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:20.918938 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eae62-0787-439f-bf19-b9d0820b6a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:20.919557 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eae62-0787-439f-bf19-b9d0820b6a7c None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:20.919845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eae62-0787-439f-bf19-b9d0820b6a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:20.920128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eae62-0787-439f-bf19-b9d0820b6a7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:20.927030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:20.931088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7eae62-0787-439f-bf19-b9d0820b6a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-ee7eae62-0787-439f-bf19-b9d0820b6a7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:45:20.936618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:20.937577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:20.938841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 449/926] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Oct 14 17:45:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:20.943825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648ab75-c357-413f-8b63-f63f29e41bca None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:21.000641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:21.051572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648ab75-c357-413f-8b63-f63f29e41bca None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:21.102003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73736458-1e0b-4e4b-82c5-2ee32d21e558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:21.102787 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73736458-1e0b-4e4b-82c5-2ee32d21e558 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:21.103064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73736458-1e0b-4e4b-82c5-2ee32d21e558 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:21.103332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73736458-1e0b-4e4b-82c5-2ee32d21e558 None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:21.104462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73736458-1e0b-4e4b-82c5-2ee32d21e558 None admin] RBAC: Adding query filter params (name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:21.104866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73736458-1e0b-4e4b-82c5-2ee32d21e558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:21.106629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73736458-1e0b-4e4b-82c5-2ee32d21e558 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:21.117048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 478/927] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Oct 14 17:45:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:45:21.125174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eae62-0787-439f-bf19-b9d0820b6a7c None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:21.231992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eae62-0787-439f-bf19-b9d0820b6a7c None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:21.267497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094460dc-c888-4a8f-b81b-b5cc9f126044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:21.268149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094460dc-c888-4a8f-b81b-b5cc9f126044 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:21.268421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094460dc-c888-4a8f-b81b-b5cc9f126044 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:21.268687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094460dc-c888-4a8f-b81b-b5cc9f126044 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:21.279026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094460dc-c888-4a8f-b81b-b5cc9f126044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:21.280507 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094460dc-c888-4a8f-b81b-b5cc9f126044 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-094460dc-c888-4a8f-b81b-b5cc9f126044 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:45:21.295024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:21.296178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:45:21.297226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:45:21.297226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:21.297226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:21.297226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 450/928] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:21.302061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73736458-1e0b-4e4b-82c5-2ee32d21e558 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:21.411023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73736458-1e0b-4e4b-82c5-2ee32d21e558 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:21.427582 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:21.447399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7912a18-e8b1-4bfc-9439-03974522deee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:21.448781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7912a18-e8b1-4bfc-9439-03974522deee None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:21.449163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7912a18-e8b1-4bfc-9439-03974522deee None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:21.449641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7912a18-e8b1-4bfc-9439-03974522deee None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:21.450439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7912a18-e8b1-4bfc-9439-03974522deee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:21.450851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7912a18-e8b1-4bfc-9439-03974522deee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:21.452337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7912a18-e8b1-4bfc-9439-03974522deee None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:21.464581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 479/929] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:21.470522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094460dc-c888-4a8f-b81b-b5cc9f126044 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:21.529693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:21.592663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094460dc-c888-4a8f-b81b-b5cc9f126044 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:21.629461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c3b34-21f3-41d4-ab45-50c1f0e7fa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:21.630143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3b34-21f3-41d4-ab45-50c1f0e7fa9f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:21.630422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3b34-21f3-41d4-ab45-50c1f0e7fa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:21.630702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3b34-21f3-41d4-ab45-50c1f0e7fa9f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:21.638912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c3b34-21f3-41d4-ab45-50c1f0e7fa9f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:21.640568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c3b34-21f3-41d4-ab45-50c1f0e7fa9f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-8c9c3b34-21f3-41d4-ab45-50c1f0e7fa9f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:45:21.653160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:21.659021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:21.660242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 451/930] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:45:21.660985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7912a18-e8b1-4bfc-9439-03974522deee None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:21.769201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7912a18-e8b1-4bfc-9439-03974522deee None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:21.804314 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa27f4-baef-4998-8476-c8360a6c7088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:21.804907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa27f4-baef-4998-8476-c8360a6c7088 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:21.805176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa27f4-baef-4998-8476-c8360a6c7088 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:21.805438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa27f4-baef-4998-8476-c8360a6c7088 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:21.806150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa27f4-baef-4998-8476-c8360a6c7088 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:21.806444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa27f4-baef-4998-8476-c8360a6c7088 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:21.808820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa27f4-baef-4998-8476-c8360a6c7088 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:21.826363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 480/931] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Oct 14 17:45:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:45:21.833510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c3b34-21f3-41d4-ab45-50c1f0e7fa9f None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:21.942907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c3b34-21f3-41d4-ab45-50c1f0e7fa9f None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:21.954300 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:21.978244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6e71a-2b4f-4efa-9fc3-c7085aa6fe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:21.978844 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6e71a-2b4f-4efa-9fc3-c7085aa6fe42 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:21.979856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6e71a-2b4f-4efa-9fc3-c7085aa6fe42 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:21.980077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6e71a-2b4f-4efa-9fc3-c7085aa6fe42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:22.054017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6e71a-2b4f-4efa-9fc3-c7085aa6fe42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:22.055738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6e71a-2b4f-4efa-9fc3-c7085aa6fe42 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:22.167191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa27f4-baef-4998-8476-c8360a6c7088 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:22.167636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 452/932] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:45:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:45:22.203290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:22.276577 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa27f4-baef-4998-8476-c8360a6c7088 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:22.313144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23934a3-a87f-4442-9791-a3a941396b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:22.313818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23934a3-a87f-4442-9791-a3a941396b27 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:22.314170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23934a3-a87f-4442-9791-a3a941396b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:22.314533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23934a3-a87f-4442-9791-a3a941396b27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:22.324936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23934a3-a87f-4442-9791-a3a941396b27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:22.326514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23934a3-a87f-4442-9791-a3a941396b27 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-d23934a3-a87f-4442-9791-a3a941396b27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:45:22.341677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:22.347474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:45:22.348488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:45:22.348488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:22.348488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:22.348488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 481/933] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:22.349812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6e71a-2b4f-4efa-9fc3-c7085aa6fe42 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:22.362181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:22.460033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6e71a-2b4f-4efa-9fc3-c7085aa6fe42 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:22.494372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c396954-809d-4b7a-a1db-652138372d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:22.495260 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c396954-809d-4b7a-a1db-652138372d64 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:22.495371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c396954-809d-4b7a-a1db-652138372d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:22.495612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c396954-809d-4b7a-a1db-652138372d64 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:22.496279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c396954-809d-4b7a-a1db-652138372d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:22.497095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c396954-809d-4b7a-a1db-652138372d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:22.497942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c396954-809d-4b7a-a1db-652138372d64 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:22.509307 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 453/934] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:22.515634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23934a3-a87f-4442-9791-a3a941396b27 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:22.606625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:22.625279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23934a3-a87f-4442-9791-a3a941396b27 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:22.658733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a06c9-cf26-45cc-9734-71c7ca4f41cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:22.659346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a06c9-cf26-45cc-9734-71c7ca4f41cf None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:22.659615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a06c9-cf26-45cc-9734-71c7ca4f41cf None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:22.659899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a06c9-cf26-45cc-9734-71c7ca4f41cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:22.666667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a06c9-cf26-45cc-9734-71c7ca4f41cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:22.668864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a06c9-cf26-45cc-9734-71c7ca4f41cf None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-034a06c9-cf26-45cc-9734-71c7ca4f41cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:45:22.680436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:22.681707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:22.683635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:45:22.683635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:45:22.683635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:45:22.683635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:45:22.683635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:45:22.683635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:45:22.683635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:45:22.683635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:22.683635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 482/935] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:22.688243 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c396954-809d-4b7a-a1db-652138372d64 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:22.768445 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:22.802459 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c396954-809d-4b7a-a1db-652138372d64 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:22.836827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b66e15-df75-4bcf-8ead-d0a00e91e204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:22.837440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b66e15-df75-4bcf-8ead-d0a00e91e204 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:22.837716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b66e15-df75-4bcf-8ead-d0a00e91e204 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:22.837984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b66e15-df75-4bcf-8ead-d0a00e91e204 None admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:22.839207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b66e15-df75-4bcf-8ead-d0a00e91e204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:22.839533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b66e15-df75-4bcf-8ead-d0a00e91e204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:22.840989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b66e15-df75-4bcf-8ead-d0a00e91e204 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:22.853478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 454/936] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:45:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:45:22.860748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a06c9-cf26-45cc-9734-71c7ca4f41cf None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:22.976055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a06c9-cf26-45cc-9734-71c7ca4f41cf None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:23.019075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f0ba6-bc47-42dc-8e1a-977a0fe7e90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:23.019595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f0ba6-bc47-42dc-8e1a-977a0fe7e90c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:23.019866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f0ba6-bc47-42dc-8e1a-977a0fe7e90c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:23.020159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f0ba6-bc47-42dc-8e1a-977a0fe7e90c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:23.020878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f0ba6-bc47-42dc-8e1a-977a0fe7e90c None admin] RBAC: Adding query filter params (role.id=e1a3bea83ac349acb6d65b6afb5441d6, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=73089bc9f85a405b86c4cd3f8cfa830a) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:23.021700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f0ba6-bc47-42dc-8e1a-977a0fe7e90c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:23.023124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f0ba6-bc47-42dc-8e1a-977a0fe7e90c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:23.030524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 483/937] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:45:22 2022] GET /identity/v3/role_assignments?user.id=73089bc9f85a405b86c4cd3f8cfa830a&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=e1a3bea83ac349acb6d65b6afb5441d6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:25.213952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea1470-6c93-4e68-b478-1495321657ee None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:25.214707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea1470-6c93-4e68-b478-1495321657ee None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:25.214707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea1470-6c93-4e68-b478-1495321657ee None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:25.215818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 455/938] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:45:25.221139 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79e9d0-e24f-4178-975f-79e8235eb154 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:25.221561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79e9d0-e24f-4178-975f-79e8235eb154 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:25.221682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79e9d0-e24f-4178-975f-79e8235eb154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:25.240391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0d79e9d0-e24f-4178-975f-79e8235eb154 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:45:25.241677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0d79e9d0-e24f-4178-975f-79e8235eb154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:45:25.244594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:25.311501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 484/939] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:45:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 17:45:25.581419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea1470-6c93-4e68-b478-1495321657ee None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:25.691997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea1470-6c93-4e68-b478-1495321657ee None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:25.724108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:25.730854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965ed6d-6232-4e33-9fde-e522f3e710e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:25.731488 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965ed6d-6232-4e33-9fde-e522f3e710e9 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:25.731770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965ed6d-6232-4e33-9fde-e522f3e710e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:25.732079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965ed6d-6232-4e33-9fde-e522f3e710e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:25.743082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965ed6d-6232-4e33-9fde-e522f3e710e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-1965ed6d-6232-4e33-9fde-e522f3e710e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:45:25.747791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:25.748922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:25.750002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 456/940] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:45:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:25.755444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79e9d0-e24f-4178-975f-79e8235eb154 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:25.866201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79e9d0-e24f-4178-975f-79e8235eb154 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:25.899976 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996e6b6-6423-401f-b504-2d7f351e65de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:25.900499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996e6b6-6423-401f-b504-2d7f351e65de None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:25.900769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996e6b6-6423-401f-b504-2d7f351e65de None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:25.901032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996e6b6-6423-401f-b504-2d7f351e65de None admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:25.902696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996e6b6-6423-401f-b504-2d7f351e65de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:25.903016 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996e6b6-6423-401f-b504-2d7f351e65de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:25.904418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996e6b6-6423-401f-b504-2d7f351e65de None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:25.912604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 485/941] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:45:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:25.919761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965ed6d-6232-4e33-9fde-e522f3e710e9 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:26.028775 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965ed6d-6232-4e33-9fde-e522f3e710e9 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:26.067656 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597030c7-0d98-4164-9bf1-fef0d5917bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:26.068392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597030c7-0d98-4164-9bf1-fef0d5917bde None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:26.068589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597030c7-0d98-4164-9bf1-fef0d5917bde None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:26.068894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597030c7-0d98-4164-9bf1-fef0d5917bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:26.080367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597030c7-0d98-4164-9bf1-fef0d5917bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:26.082361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597030c7-0d98-4164-9bf1-fef0d5917bde None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-597030c7-0d98-4164-9bf1-fef0d5917bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:45:26.101555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:26.102826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:45:26.104141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:45:26.104141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:26.104141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:26.104141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 457/942] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:26.110879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996e6b6-6423-401f-b504-2d7f351e65de None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:26.114981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:26.226405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996e6b6-6423-401f-b504-2d7f351e65de None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:26.264286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1482c2e-00e7-452e-a01e-eb74ff246d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:26.264893 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1482c2e-00e7-452e-a01e-eb74ff246d42 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:26.265208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1482c2e-00e7-452e-a01e-eb74ff246d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:26.265696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1482c2e-00e7-452e-a01e-eb74ff246d42 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:26.266579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1482c2e-00e7-452e-a01e-eb74ff246d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:26.267076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1482c2e-00e7-452e-a01e-eb74ff246d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:26.268748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1482c2e-00e7-452e-a01e-eb74ff246d42 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:26.284740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 486/943] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:45:26.292242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597030c7-0d98-4164-9bf1-fef0d5917bde None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:26.341480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:26.411455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597030c7-0d98-4164-9bf1-fef0d5917bde None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:26.447281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a39d8-2af1-4d2a-8359-968c5d823452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:26.447937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a39d8-2af1-4d2a-8359-968c5d823452 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:26.448774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a39d8-2af1-4d2a-8359-968c5d823452 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:26.449047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a39d8-2af1-4d2a-8359-968c5d823452 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:26.457092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a39d8-2af1-4d2a-8359-968c5d823452 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:26.458923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a39d8-2af1-4d2a-8359-968c5d823452 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-107a39d8-2af1-4d2a-8359-968c5d823452 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:45:26.470888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:26.472006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:26.473270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 458/944] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:26.478648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1482c2e-00e7-452e-a01e-eb74ff246d42 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:26.545727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:26.589543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1482c2e-00e7-452e-a01e-eb74ff246d42 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:26.631644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a9a56-70b3-482c-a048-951d14563714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:26.632276 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a9a56-70b3-482c-a048-951d14563714 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:26.632541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a9a56-70b3-482c-a048-951d14563714 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:26.632807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a9a56-70b3-482c-a048-951d14563714 None admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:26.633525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a9a56-70b3-482c-a048-951d14563714 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:26.633824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a9a56-70b3-482c-a048-951d14563714 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:26.635260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a9a56-70b3-482c-a048-951d14563714 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:26.654051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 487/945] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Oct 14 17:45:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:26.659888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a39d8-2af1-4d2a-8359-968c5d823452 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:26.771020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a39d8-2af1-4d2a-8359-968c5d823452 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:26.772793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:26.811705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b471cad-3b61-408f-ac12-c1a3692870c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:26.812357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b471cad-3b61-408f-ac12-c1a3692870c4 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:26.812633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b471cad-3b61-408f-ac12-c1a3692870c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:26.812897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b471cad-3b61-408f-ac12-c1a3692870c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:26.824631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b471cad-3b61-408f-ac12-c1a3692870c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:26.826071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b471cad-3b61-408f-ac12-c1a3692870c4 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-2b471cad-3b61-408f-ac12-c1a3692870c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:45:26.842141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:26.843320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:45:26.844659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:45:26.844659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:26.844659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:26.844659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 459/946] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:26.849179 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a9a56-70b3-482c-a048-951d14563714 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:26.958669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a9a56-70b3-482c-a048-951d14563714 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:26.981116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:26.997406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a77ca1-d1ed-4896-9d12-de8c779baa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:26.998007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a77ca1-d1ed-4896-9d12-de8c779baa36 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:26.998279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a77ca1-d1ed-4896-9d12-de8c779baa36 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:26.998576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a77ca1-d1ed-4896-9d12-de8c779baa36 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:26.999180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a77ca1-d1ed-4896-9d12-de8c779baa36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:26.999473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a77ca1-d1ed-4896-9d12-de8c779baa36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:27.000883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a77ca1-d1ed-4896-9d12-de8c779baa36 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:27.012247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 488/947] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:27.017987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b471cad-3b61-408f-ac12-c1a3692870c4 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:27.147278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b471cad-3b61-408f-ac12-c1a3692870c4 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:27.189161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c6975-08da-4149-a96c-0565a1e51928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:27.189941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c6975-08da-4149-a96c-0565a1e51928 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:27.190330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c6975-08da-4149-a96c-0565a1e51928 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:27.190805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c6975-08da-4149-a96c-0565a1e51928 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:27.200118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c6975-08da-4149-a96c-0565a1e51928 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:27.201919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c6975-08da-4149-a96c-0565a1e51928 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-ca5c6975-08da-4149-a96c-0565a1e51928 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:45:27.213456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:27.214802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:27.215709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:45:27.215709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:45:27.215709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:45:27.215709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:45:27.215709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:45:27.215709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:45:27.215709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:45:27.215709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:27.215709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 460/948] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:45:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:27.220616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a77ca1-d1ed-4896-9d12-de8c779baa36 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:27.326930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a77ca1-d1ed-4896-9d12-de8c779baa36 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:27.363596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a46af-4f6c-48d4-8e10-46469e4d244b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:27.364236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a46af-4f6c-48d4-8e10-46469e4d244b None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:27.364509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a46af-4f6c-48d4-8e10-46469e4d244b None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:27.364773 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a46af-4f6c-48d4-8e10-46469e4d244b None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:27.365457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a46af-4f6c-48d4-8e10-46469e4d244b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:27.365769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a46af-4f6c-48d4-8e10-46469e4d244b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:27.367180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a46af-4f6c-48d4-8e10-46469e4d244b None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:27.379289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 489/949] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:45:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:27.385458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c6975-08da-4149-a96c-0565a1e51928 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:27.405891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:27.498970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c6975-08da-4149-a96c-0565a1e51928 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:27.537638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95a983-686b-4574-8701-fa34a1e958c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:27.538398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a983-686b-4574-8701-fa34a1e958c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:27.538591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a983-686b-4574-8701-fa34a1e958c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:27.538848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a983-686b-4574-8701-fa34a1e958c5 None admin] PATH_INFO: `/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/73089bc9f85a405b86c4cd3f8cfa830a/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:27.540769 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95a983-686b-4574-8701-fa34a1e958c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=009e40c8d9ec4f828b2ee0f735842aa0, user_id=73089bc9f85a405b86c4cd3f8cfa830a, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:27.542480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95a983-686b-4574-8701-fa34a1e958c5 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:27.555234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 461/950] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Oct 14 17:45:27 2022] PUT /identity/v3/projects/009e40c8d9ec4f828b2ee0f735842aa0/users/73089bc9f85a405b86c4cd3f8cfa830a/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:45:29.836436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c334f-c532-46a4-8dec-c7a0fa379407 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:29.836999 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c334f-c532-46a4-8dec-c7a0fa379407 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:29.837109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c334f-c532-46a4-8dec-c7a0fa379407 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:29.838623 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 490/951] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:45:29.845670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d464710-4c9a-414f-96ca-d0952102db72 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:29.846111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d464710-4c9a-414f-96ca-d0952102db72 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:29.846201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d464710-4c9a-414f-96ca-d0952102db72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:29.866673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2d464710-4c9a-414f-96ca-d0952102db72 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:45:29.867936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2d464710-4c9a-414f-96ca-d0952102db72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:45:29.949337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 462/952] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:45:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:45:30.222095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c334f-c532-46a4-8dec-c7a0fa379407 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:30.272310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:30.340258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c334f-c532-46a4-8dec-c7a0fa379407 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:30.385343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29227c6e-5e7c-43b5-aa27-6710ae7555e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:30.386854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29227c6e-5e7c-43b5-aa27-6710ae7555e7 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:30.387156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29227c6e-5e7c-43b5-aa27-6710ae7555e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:30.387671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29227c6e-5e7c-43b5-aa27-6710ae7555e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:30.406102 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29227c6e-5e7c-43b5-aa27-6710ae7555e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-29227c6e-5e7c-43b5-aa27-6710ae7555e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 17:45:30.411313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:30.412489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:30.413351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 491/953] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Oct 14 17:45:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:45:30.419389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d464710-4c9a-414f-96ca-d0952102db72 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:30.423064 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:30.534512 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d464710-4c9a-414f-96ca-d0952102db72 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:30.578628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6723ac5-ad2b-43d4-9de8-fb7e59caf64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:30.579387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6723ac5-ad2b-43d4-9de8-fb7e59caf64e None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:30.579736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6723ac5-ad2b-43d4-9de8-fb7e59caf64e None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:30.580052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6723ac5-ad2b-43d4-9de8-fb7e59caf64e None admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:30.580849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6723ac5-ad2b-43d4-9de8-fb7e59caf64e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:30.581237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6723ac5-ad2b-43d4-9de8-fb7e59caf64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:30.582780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6723ac5-ad2b-43d4-9de8-fb7e59caf64e None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:30.593599 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 463/954] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Oct 14 17:45:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:30.599698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29227c6e-5e7c-43b5-aa27-6710ae7555e7 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:30.713708 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:30.717304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29227c6e-5e7c-43b5-aa27-6710ae7555e7 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:30.759468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74712f8e-719a-4376-b1db-bff67774ab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:30.760245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74712f8e-719a-4376-b1db-bff67774ab94 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:30.760571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74712f8e-719a-4376-b1db-bff67774ab94 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:30.761231 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74712f8e-719a-4376-b1db-bff67774ab94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:30.772699 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74712f8e-719a-4376-b1db-bff67774ab94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:30.774137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74712f8e-719a-4376-b1db-bff67774ab94 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-74712f8e-719a-4376-b1db-bff67774ab94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:45:30.790487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:30.791635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:45:30.793509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:45:30.793509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:30.793509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:30.797715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 492/955] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:30.799240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6723ac5-ad2b-43d4-9de8-fb7e59caf64e None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:30.873614 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:30.914102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6723ac5-ad2b-43d4-9de8-fb7e59caf64e None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:30.954507 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dafe3-297c-4645-8285-89da0b5f73c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:30.955110 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dafe3-297c-4645-8285-89da0b5f73c7 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:30.955383 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dafe3-297c-4645-8285-89da0b5f73c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:30.955649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dafe3-297c-4645-8285-89da0b5f73c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:30.956320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dafe3-297c-4645-8285-89da0b5f73c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:30.956633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dafe3-297c-4645-8285-89da0b5f73c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:30.957979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dafe3-297c-4645-8285-89da0b5f73c7 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:30.970068 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 464/956] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:30.975758 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74712f8e-719a-4376-b1db-bff67774ab94 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:31.098900 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74712f8e-719a-4376-b1db-bff67774ab94 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:31.143603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c38d20-9b31-4e38-9a5a-cd9a49c976ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:31.144258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c38d20-9b31-4e38-9a5a-cd9a49c976ab None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:31.144548 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c38d20-9b31-4e38-9a5a-cd9a49c976ab None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:31.144795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c38d20-9b31-4e38-9a5a-cd9a49c976ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:31.153503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c38d20-9b31-4e38-9a5a-cd9a49c976ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:31.154934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c38d20-9b31-4e38-9a5a-cd9a49c976ab None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-84c38d20-9b31-4e38-9a5a-cd9a49c976ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 17:45:31.167468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:31.168528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:31.169566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 493/957] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:31.174523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dafe3-297c-4645-8285-89da0b5f73c7 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:31.283725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dafe3-297c-4645-8285-89da0b5f73c7 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:31.312834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:31.323015 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abceb4-3015-4fc5-8962-466d7951fbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:31.323899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abceb4-3015-4fc5-8962-466d7951fbfe None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:31.324188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abceb4-3015-4fc5-8962-466d7951fbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:31.324462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abceb4-3015-4fc5-8962-466d7951fbfe None admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:31.325175 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abceb4-3015-4fc5-8962-466d7951fbfe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:31.325478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abceb4-3015-4fc5-8962-466d7951fbfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:31.326884 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abceb4-3015-4fc5-8962-466d7951fbfe None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:31.345290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 465/958] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Oct 14 17:45:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:31.351533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c38d20-9b31-4e38-9a5a-cd9a49c976ab None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:31.370186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:31.461931 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c38d20-9b31-4e38-9a5a-cd9a49c976ab None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:31.501406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19092aea-d508-4a8a-a465-05fb97ea93fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:31.502030 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19092aea-d508-4a8a-a465-05fb97ea93fa None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:31.502301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19092aea-d508-4a8a-a465-05fb97ea93fa None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:31.502562 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19092aea-d508-4a8a-a465-05fb97ea93fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:31.571577 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19092aea-d508-4a8a-a465-05fb97ea93fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:31.573128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19092aea-d508-4a8a-a465-05fb97ea93fa None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:31.586820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:31.686761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 494/959] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Oct 14 17:45:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 17:45:31.692700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abceb4-3015-4fc5-8962-466d7951fbfe None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:31.809770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abceb4-3015-4fc5-8962-466d7951fbfe None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:31.845942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdb680-02ef-4d82-bd5a-52c7bc44d864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:31.846532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdb680-02ef-4d82-bd5a-52c7bc44d864 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:31.846805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdb680-02ef-4d82-bd5a-52c7bc44d864 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:31.847072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdb680-02ef-4d82-bd5a-52c7bc44d864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:31.858782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdb680-02ef-4d82-bd5a-52c7bc44d864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:31.860498 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdb680-02ef-4d82-bd5a-52c7bc44d864 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-0dcdb680-02ef-4d82-bd5a-52c7bc44d864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 17:45:31.877574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:31.879029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 17:45:31.880312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 17:45:31.880312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 17:45:31.880312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:31.880312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 466/960] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 17:45:31.884347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19092aea-d508-4a8a-a465-05fb97ea93fa None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:31.998882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19092aea-d508-4a8a-a465-05fb97ea93fa None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:32.014289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:32.038403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43fefe-452f-4ff8-97a7-4e9bfe7c1ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:32.039016 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43fefe-452f-4ff8-97a7-4e9bfe7c1ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:32.039287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43fefe-452f-4ff8-97a7-4e9bfe7c1ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:32.039549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43fefe-452f-4ff8-97a7-4e9bfe7c1ac0 None admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:32.040220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43fefe-452f-4ff8-97a7-4e9bfe7c1ac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:32.040512 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43fefe-452f-4ff8-97a7-4e9bfe7c1ac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:32.041879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43fefe-452f-4ff8-97a7-4e9bfe7c1ac0 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:32.054923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 495/961] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Oct 14 17:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:32.061688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdb680-02ef-4d82-bd5a-52c7bc44d864 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:32.105623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:32.185587 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdb680-02ef-4d82-bd5a-52c7bc44d864 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:32.223848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8822c-9c0c-4f1c-a7cc-78f7b34e9b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:32.224522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8822c-9c0c-4f1c-a7cc-78f7b34e9b55 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:32.224803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8822c-9c0c-4f1c-a7cc-78f7b34e9b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:32.225071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8822c-9c0c-4f1c-a7cc-78f7b34e9b55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:32.232830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8822c-9c0c-4f1c-a7cc-78f7b34e9b55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:32.234307 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8822c-9c0c-4f1c-a7cc-78f7b34e9b55 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application [None req-2ad8822c-9c0c-4f1c-a7cc-78f7b34e9b55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 17:45:32.246497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:32.247645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:32.249090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 17:45:32.249090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 17:45:32.249090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 17:45:32.249090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 17:45:32.249090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 17:45:32.249090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 17:45:32.249090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 17:45:32.249090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: ERROR keystone.server.flask.application Oct 14 17:45:32.252044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 467/962] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Oct 14 17:45:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:32.253617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43fefe-452f-4ff8-97a7-4e9bfe7c1ac0 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:32.366271 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43fefe-452f-4ff8-97a7-4e9bfe7c1ac0 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:32.402419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758868d8-c112-4482-83e9-825b89483a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:32.403031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758868d8-c112-4482-83e9-825b89483a73 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:32.403300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758868d8-c112-4482-83e9-825b89483a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:32.403572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758868d8-c112-4482-83e9-825b89483a73 None admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:32.404287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758868d8-c112-4482-83e9-825b89483a73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:32.404586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758868d8-c112-4482-83e9-825b89483a73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:32.406017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758868d8-c112-4482-83e9-825b89483a73 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:32.418943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 496/963] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Oct 14 17:45:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:32.424985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8822c-9c0c-4f1c-a7cc-78f7b34e9b55 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:32.532908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:32.539134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8822c-9c0c-4f1c-a7cc-78f7b34e9b55 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:32.575262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be451f4-25c6-4425-939b-b663f4d40b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:32.575853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be451f4-25c6-4425-939b-b663f4d40b4d None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:32.576153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be451f4-25c6-4425-939b-b663f4d40b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:32.576417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be451f4-25c6-4425-939b-b663f4d40b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:32.577132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be451f4-25c6-4425-939b-b663f4d40b4d None admin] RBAC: Adding query filter params (role.id=45b67ba9922c4b16baae3930d3de1b6c, scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0, user.id=73089bc9f85a405b86c4cd3f8cfa830a) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:32.577436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be451f4-25c6-4425-939b-b663f4d40b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:32.578849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be451f4-25c6-4425-939b-b663f4d40b4d None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:32.585152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 468/964] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Oct 14 17:45:32 2022] GET /identity/v3/role_assignments?user.id=73089bc9f85a405b86c4cd3f8cfa830a&scope.project.id=009e40c8d9ec4f828b2ee0f735842aa0&role.id=45b67ba9922c4b16baae3930d3de1b6c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:34.857713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf3fd3-1f6c-4da7-835a-82dcd24a5f72 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:34.858232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf3fd3-1f6c-4da7-835a-82dcd24a5f72 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:34.858232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf3fd3-1f6c-4da7-835a-82dcd24a5f72 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:34.859549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 497/965] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:45:34.864725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2055a8-f9c9-4e4c-b0af-d13f7c37e714 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:34.865167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2055a8-f9c9-4e4c-b0af-d13f7c37e714 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:34.865262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2055a8-f9c9-4e4c-b0af-d13f7c37e714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:34.882761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-af2055a8-f9c9-4e4c-b0af-d13f7c37e714 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:45:34.883955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-af2055a8-f9c9-4e4c-b0af-d13f7c37e714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:45:34.949662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 469/966] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:45:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:45:35.246601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf3fd3-1f6c-4da7-835a-82dcd24a5f72 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:35.277632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:35.347195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf3fd3-1f6c-4da7-835a-82dcd24a5f72 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:35.380839 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a5d3d-c2d9-46e6-ac42-c8f02721e4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:35.381452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a5d3d-c2d9-46e6-ac42-c8f02721e4bc None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:35.381723 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a5d3d-c2d9-46e6-ac42-c8f02721e4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:35.381987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a5d3d-c2d9-46e6-ac42-c8f02721e4bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:35.382553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a5d3d-c2d9-46e6-ac42-c8f02721e4bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:35.383935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a5d3d-c2d9-46e6-ac42-c8f02721e4bc None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-5e4a5d3d-c2d9-46e6-ac42-c8f02721e4bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:35.393500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:35.394754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:35.395627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 498/967] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Oct 14 17:45:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:35.400778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2055a8-f9c9-4e4c-b0af-d13f7c37e714 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:35.486827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:35.497506 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2055a8-f9c9-4e4c-b0af-d13f7c37e714 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:35.530880 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcec34-8d91-4738-96a4-0214bee08d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:35.531480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcec34-8d91-4738-96a4-0214bee08d8f None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:35.531752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcec34-8d91-4738-96a4-0214bee08d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:35.532053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcec34-8d91-4738-96a4-0214bee08d8f None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:35.532665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcec34-8d91-4738-96a4-0214bee08d8f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:35.532959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcec34-8d91-4738-96a4-0214bee08d8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:35.534305 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcec34-8d91-4738-96a4-0214bee08d8f None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:35.538913 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 470/968] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Oct 14 17:45:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:35.544874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a5d3d-c2d9-46e6-ac42-c8f02721e4bc None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:35.635182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:35.642362 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a5d3d-c2d9-46e6-ac42-c8f02721e4bc None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:35.674274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a1ebb-852f-4752-b806-ff7016dd522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:35.674853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1ebb-852f-4752-b806-ff7016dd522a None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:35.675123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1ebb-852f-4752-b806-ff7016dd522a None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:35.675384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1ebb-852f-4752-b806-ff7016dd522a None admin] PATH_INFO: `/v3/services` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:35.676022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a1ebb-852f-4752-b806-ff7016dd522a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:35.676313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a1ebb-852f-4752-b806-ff7016dd522a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:35.677656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a1ebb-852f-4752-b806-ff7016dd522a None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:35.681843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 499/969] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Oct 14 17:45:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:37.856142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3defd-ee37-4515-b4bf-0d8ec99292b2 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:37.856664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3defd-ee37-4515-b4bf-0d8ec99292b2 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:37.856664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3defd-ee37-4515-b4bf-0d8ec99292b2 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:37.857997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 471/970] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:45:37.863609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e67c64-b001-4f91-83f2-5690da0c9759 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:37.864056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e67c64-b001-4f91-83f2-5690da0c9759 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:37.864181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e67c64-b001-4f91-83f2-5690da0c9759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:37.883021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-30e67c64-b001-4f91-83f2-5690da0c9759 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:45:37.884242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-30e67c64-b001-4f91-83f2-5690da0c9759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:45:37.953914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 500/971] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:45:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:45:38.223705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3defd-ee37-4515-b4bf-0d8ec99292b2 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:38.321047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3defd-ee37-4515-b4bf-0d8ec99292b2 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:38.354497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3e84b-e6ab-4113-b6bb-fde0c854b89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:38.356335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3e84b-e6ab-4113-b6bb-fde0c854b89b None admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:38.356608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3e84b-e6ab-4113-b6bb-fde0c854b89b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:38.356876 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3e84b-e6ab-4113-b6bb-fde0c854b89b None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:38.357515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3e84b-e6ab-4113-b6bb-fde0c854b89b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:38.358857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3e84b-e6ab-4113-b6bb-fde0c854b89b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:38.373114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 472/972] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Oct 14 17:45:38 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:45:40.660533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9776a-f953-405c-b5bd-844d7b4b4868 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:40.661065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9776a-f953-405c-b5bd-844d7b4b4868 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:40.661065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9776a-f953-405c-b5bd-844d7b4b4868 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:40.662296 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 501/973] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:45:40.668685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935de548-b3e3-4761-812f-1f3af65e6741 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:40.669106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935de548-b3e3-4761-812f-1f3af65e6741 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:40.669285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935de548-b3e3-4761-812f-1f3af65e6741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:40.688314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-935de548-b3e3-4761-812f-1f3af65e6741 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:45:40.689604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-935de548-b3e3-4761-812f-1f3af65e6741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:45:40.699369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:40.765807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 473/974] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:45:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:45:41.063283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9776a-f953-405c-b5bd-844d7b4b4868 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:41.158573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:41.169823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9776a-f953-405c-b5bd-844d7b4b4868 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:41.203246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3855d-a2ec-4e14-8623-238234884041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:41.203860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3855d-a2ec-4e14-8623-238234884041 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:41.204167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3855d-a2ec-4e14-8623-238234884041 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:41.204433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3855d-a2ec-4e14-8623-238234884041 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:41.205002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3855d-a2ec-4e14-8623-238234884041 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:41.206370 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3855d-a2ec-4e14-8623-238234884041 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-a7b3855d-a2ec-4e14-8623-238234884041 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:41.215035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:41.216326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:41.217444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 502/975] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Oct 14 17:45:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:41.222228 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935de548-b3e3-4761-812f-1f3af65e6741 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:41.318489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935de548-b3e3-4761-812f-1f3af65e6741 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:41.351736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc38cd-1d19-46c6-b7a3-ec735fd48d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:41.352361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc38cd-1d19-46c6-b7a3-ec735fd48d44 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:41.352634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc38cd-1d19-46c6-b7a3-ec735fd48d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:41.352898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc38cd-1d19-46c6-b7a3-ec735fd48d44 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:41.353509 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc38cd-1d19-46c6-b7a3-ec735fd48d44 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:41.353805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc38cd-1d19-46c6-b7a3-ec735fd48d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:41.355786 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc38cd-1d19-46c6-b7a3-ec735fd48d44 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:41.360819 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 474/976] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Oct 14 17:45:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:41.367025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3855d-a2ec-4e14-8623-238234884041 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:41.471356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3855d-a2ec-4e14-8623-238234884041 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:41.507719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fa286-c05e-4e47-bf4a-0e5f782f290c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:41.508445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fa286-c05e-4e47-bf4a-0e5f782f290c None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:41.508755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fa286-c05e-4e47-bf4a-0e5f782f290c None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:41.509033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fa286-c05e-4e47-bf4a-0e5f782f290c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:41.509887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fa286-c05e-4e47-bf4a-0e5f782f290c None admin] RBAC: Adding query filter params (interface=public, service_id=c6f42d25677b45c6af550d2fa1917b2a, region_id=RegionOne) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:41.510195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fa286-c05e-4e47-bf4a-0e5f782f290c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:41.511853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fa286-c05e-4e47-bf4a-0e5f782f290c None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:41.517292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 503/977] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Oct 14 17:45:41 2022] GET /identity/v3/endpoints?service_id=c6f42d25677b45c6af550d2fa1917b2a&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:41.523150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc38cd-1d19-46c6-b7a3-ec735fd48d44 None admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:41.527391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:41.622959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc38cd-1d19-46c6-b7a3-ec735fd48d44 None admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:41.655749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e8325-4dd5-4ebe-b75f-b54c4a02386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:41.656349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e8325-4dd5-4ebe-b75f-b54c4a02386b None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:41.656619 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e8325-4dd5-4ebe-b75f-b54c4a02386b None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:41.656881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e8325-4dd5-4ebe-b75f-b54c4a02386b None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:41.657464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e8325-4dd5-4ebe-b75f-b54c4a02386b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:41.658799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e8325-4dd5-4ebe-b75f-b54c4a02386b None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:41.665279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 475/978] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Oct 14 17:45:41 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:45:43.790770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba69e5-2532-437e-9a2e-b35aa952d0c6 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:43.791281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba69e5-2532-437e-9a2e-b35aa952d0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:43.791281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba69e5-2532-437e-9a2e-b35aa952d0c6 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:43.793841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 504/979] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:45:43.799088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d5ea6-88af-416f-b861-67666e4a2e02 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:43.799468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d5ea6-88af-416f-b861-67666e4a2e02 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:43.799622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d5ea6-88af-416f-b861-67666e4a2e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:43.817337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1e4d5ea6-88af-416f-b861-67666e4a2e02 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:45:43.818490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1e4d5ea6-88af-416f-b861-67666e4a2e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:45:43.879360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:43.885020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 476/980] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:45:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 17:45:44.154372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba69e5-2532-437e-9a2e-b35aa952d0c6 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:44.173842 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:44.252428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba69e5-2532-437e-9a2e-b35aa952d0c6 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:44.285134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d74e89-2fce-4f64-9bf7-a519e7f8b779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:44.285729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74e89-2fce-4f64-9bf7-a519e7f8b779 None admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:44.286001 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74e89-2fce-4f64-9bf7-a519e7f8b779 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:44.286267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74e89-2fce-4f64-9bf7-a519e7f8b779 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:44.286832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d74e89-2fce-4f64-9bf7-a519e7f8b779 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:44.288209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d74e89-2fce-4f64-9bf7-a519e7f8b779 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application [None req-03d74e89-2fce-4f64-9bf7-a519e7f8b779 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 17:45:44.296821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 17:45:44.298169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application with Lock( Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self._enter() Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application return self.creator() Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application created_value = creator( Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: ERROR keystone.server.flask.application Oct 14 17:45:44.299335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 505/981] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Oct 14 17:45:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 17:45:44.303863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d5ea6-88af-416f-b861-67666e4a2e02 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:44.409256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d5ea6-88af-416f-b861-67666e4a2e02 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:44.444700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036d649-3a1d-45cd-a6ba-f1be1c42e2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:44.445349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036d649-3a1d-45cd-a6ba-f1be1c42e2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:44.445673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036d649-3a1d-45cd-a6ba-f1be1c42e2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:44.445955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036d649-3a1d-45cd-a6ba-f1be1c42e2f3 None admin] PATH_INFO: `/v3/services` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:44.446629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036d649-3a1d-45cd-a6ba-f1be1c42e2f3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:45:44.446983 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036d649-3a1d-45cd-a6ba-f1be1c42e2f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:44.448434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036d649-3a1d-45cd-a6ba-f1be1c42e2f3 None admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:44.454613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 477/982] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Oct 14 17:45:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:45:44.460332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d74e89-2fce-4f64-9bf7-a519e7f8b779 None admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:45:44.538990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:44.556681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d74e89-2fce-4f64-9bf7-a519e7f8b779 None admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:45:44.590645 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a2f54-14dc-4dbb-823b-c2485a42c3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:45:44.591220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a2f54-14dc-4dbb-823b-c2485a42c3d6 None admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:45:44.591491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a2f54-14dc-4dbb-823b-c2485a42c3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:45:44.591755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a2f54-14dc-4dbb-823b-c2485a42c3d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:45:44.592435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a2f54-14dc-4dbb-823b-c2485a42c3d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:45:44.593783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a2f54-14dc-4dbb-823b-c2485a42c3d6 None admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:45:44.612112 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 506/983] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:45:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:45:54.451839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=69 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:45:54.608259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:46:48.875059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dac879-7736-4e29-a4bf-184ed5599eee None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:48.875527 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dac879-7736-4e29-a4bf-184ed5599eee None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:48.875756 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dac879-7736-4e29-a4bf-184ed5599eee None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:48.877206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 478/984] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:46:48.885128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eab7a2-7afb-4a1d-a44b-5e23cba28c0e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:48.885551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eab7a2-7afb-4a1d-a44b-5e23cba28c0e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:48.885770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eab7a2-7afb-4a1d-a44b-5e23cba28c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:49.013586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-42eab7a2-7afb-4a1d-a44b-5e23cba28c0e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:46:49.015951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-42eab7a2-7afb-4a1d-a44b-5e23cba28c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:49.131054 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 507/985] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:46:49.429705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7e566-86c2-40e9-90a6-4737d093029d None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:49.429986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7e566-86c2-40e9-90a6-4737d093029d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:49.430478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7e566-86c2-40e9-90a6-4737d093029d None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:49.431798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 479/986] 10.222.0.27 () {54 vars in 964 bytes} [Fri Oct 14 17:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:46:49.438491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224df63-4642-4bb0-bd7f-664972e57cd8 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:49.438771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224df63-4642-4bb0-bd7f-664972e57cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:49.439050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224df63-4642-4bb0-bd7f-664972e57cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:49.519684 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-b224df63-4642-4bb0-bd7f-664972e57cd8 None None] MFA Rules not processed for user `3c17f274e4c14f2f99650844642437a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:46:49.520956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b224df63-4642-4bb0-bd7f-664972e57cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:49.542838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:46:49.604401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 508/987] 10.222.0.27 () {58 vars in 1061 bytes} [Fri Oct 14 17:46:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:46:49.614041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7e566-86c2-40e9-90a6-4737d093029d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:46:49.615768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-66a7e566-86c2-40e9-90a6-4737d093029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:49.739975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7e566-86c2-40e9-90a6-4737d093029d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:46:49.822804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450c806-65d8-4b16-9160-eed7ff8dfa77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:46:49.823585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450c806-65d8-4b16-9160-eed7ff8dfa77 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:49.823984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450c806-65d8-4b16-9160-eed7ff8dfa77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:49.824352 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450c806-65d8-4b16-9160-eed7ff8dfa77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:49.826089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c450c806-65d8-4b16-9160-eed7ff8dfa77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:49.891605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:46:49.955849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450c806-65d8-4b16-9160-eed7ff8dfa77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:46:49.957500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450c806-65d8-4b16-9160-eed7ff8dfa77 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:46:49.979574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 480/988] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:46:52.532600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad152c17-96bc-4da3-abf0-25667cb70f29 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:52.533120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad152c17-96bc-4da3-abf0-25667cb70f29 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:52.533120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad152c17-96bc-4da3-abf0-25667cb70f29 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:52.534377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 509/989] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:46:52.539499 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25874b-e03a-42f7-add6-ce13bbb919d0 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:52.539909 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25874b-e03a-42f7-add6-ce13bbb919d0 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:52.540072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25874b-e03a-42f7-add6-ce13bbb919d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:52.619721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1d25874b-e03a-42f7-add6-ce13bbb919d0 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:46:52.620940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1d25874b-e03a-42f7-add6-ce13bbb919d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:52.716640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 481/990] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:46:53.029453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad152c17-96bc-4da3-abf0-25667cb70f29 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:46:53.031634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ad152c17-96bc-4da3-abf0-25667cb70f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:53.132598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad152c17-96bc-4da3-abf0-25667cb70f29 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:46:53.208354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:46:53.210640 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbf30c-e04a-42ae-9041-190102f567a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:46:53.211338 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbf30c-e04a-42ae-9041-190102f567a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:53.211847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbf30c-e04a-42ae-9041-190102f567a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:53.212306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbf30c-e04a-42ae-9041-190102f567a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:53.214110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cbdbf30c-e04a-42ae-9041-190102f567a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:53.321123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbf30c-e04a-42ae-9041-190102f567a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:46:53.322550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbf30c-e04a-42ae-9041-190102f567a5 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:46:53.341328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 510/991] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:46:55.792262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef83c9-727d-4baf-8045-b44989389f7d None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:55.792791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef83c9-727d-4baf-8045-b44989389f7d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:55.792791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef83c9-727d-4baf-8045-b44989389f7d None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:55.794125 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 482/992] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:46:55.800086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623fa43-d7f1-42b4-b9c0-af7d3e1c2bc4 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:55.800479 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623fa43-d7f1-42b4-b9c0-af7d3e1c2bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:55.800632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623fa43-d7f1-42b4-b9c0-af7d3e1c2bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:55.865233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-a623fa43-d7f1-42b4-b9c0-af7d3e1c2bc4 None None] MFA Rules not processed for user `cab8f0baebc8466da5e21e19f829447a`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:46:55.866427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a623fa43-d7f1-42b4-b9c0-af7d3e1c2bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:55.918916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:46:55.963552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 511/993] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:46:55 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:46:56.246986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef83c9-727d-4baf-8045-b44989389f7d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:46:56.248568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-bfef83c9-727d-4baf-8045-b44989389f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:56.259737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:46:56.352342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef83c9-727d-4baf-8045-b44989389f7d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:46:56.427393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8af991-a1ee-45b4-9eb3-20485768bb1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:46:56.428019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8af991-a1ee-45b4-9eb3-20485768bb1b service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:56.428294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8af991-a1ee-45b4-9eb3-20485768bb1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:56.428556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8af991-a1ee-45b4-9eb3-20485768bb1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:56.430137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3a8af991-a1ee-45b4-9eb3-20485768bb1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:56.491911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:46:56.538662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8af991-a1ee-45b4-9eb3-20485768bb1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:46:56.540111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8af991-a1ee-45b4-9eb3-20485768bb1b service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:46:56.557978 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 483/994] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:46:56 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:46:59.232022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb71a7b-84fd-4a28-be34-ba8e10a64f92 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:59.232588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb71a7b-84fd-4a28-be34-ba8e10a64f92 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:59.232588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb71a7b-84fd-4a28-be34-ba8e10a64f92 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:59.234077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 512/995] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:46:59.239259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f16b0-7e89-41f2-a020-79bfa10624ee None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:59.239666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f16b0-7e89-41f2-a020-79bfa10624ee None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:59.239775 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f16b0-7e89-41f2-a020-79bfa10624ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:59.304484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-9c4f16b0-7e89-41f2-a020-79bfa10624ee None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:46:59.305668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9c4f16b0-7e89-41f2-a020-79bfa10624ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:59.400440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 484/996] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:46:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:46:59.687069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb71a7b-84fd-4a28-be34-ba8e10a64f92 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:46:59.688674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-eeb71a7b-84fd-4a28-be34-ba8e10a64f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:59.789582 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb71a7b-84fd-4a28-be34-ba8e10a64f92 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:46:59.862072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e90a4-cbbc-4e9a-b72d-550264ede273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:46:59.862662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e90a4-cbbc-4e9a-b72d-550264ede273 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:46:59.862930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e90a4-cbbc-4e9a-b72d-550264ede273 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:46:59.863192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e90a4-cbbc-4e9a-b72d-550264ede273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:46:59.864789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8c6e90a4-cbbc-4e9a-b72d-550264ede273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:46:59.871435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:46:59.971097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e90a4-cbbc-4e9a-b72d-550264ede273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:46:59.972466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e90a4-cbbc-4e9a-b72d-550264ede273 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:46:59.990354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 513/997] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:46:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:02.787329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c9802-338a-4ded-bc14-0fd79f2f10a9 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:02.787858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c9802-338a-4ded-bc14-0fd79f2f10a9 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:02.787858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c9802-338a-4ded-bc14-0fd79f2f10a9 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:02.789234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 485/998] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:02.794415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4928c-e99d-44c7-8d8c-e8b912aaa770 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:02.794843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4928c-e99d-44c7-8d8c-e8b912aaa770 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:02.795071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4928c-e99d-44c7-8d8c-e8b912aaa770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:02.865760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-cea4928c-e99d-44c7-8d8c-e8b912aaa770 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:02.867005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cea4928c-e99d-44c7-8d8c-e8b912aaa770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:02.929552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:02.972766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 514/999] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:03.257221 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c9802-338a-4ded-bc14-0fd79f2f10a9 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:03.258844 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b72c9802-338a-4ded-bc14-0fd79f2f10a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:03.279806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:03.365149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c9802-338a-4ded-bc14-0fd79f2f10a9 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:03.441312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a77650-d7c6-451c-a626-2d568421c4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:03.441937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77650-d7c6-451c-a626-2d568421c4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:03.442209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77650-d7c6-451c-a626-2d568421c4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:03.442473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77650-d7c6-451c-a626-2d568421c4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:03.444100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-88a77650-d7c6-451c-a626-2d568421c4c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:03.516672 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:03.555397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a77650-d7c6-451c-a626-2d568421c4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:03.556890 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a77650-d7c6-451c-a626-2d568421c4c8 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:03.576677 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 486/1000] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:05.804019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6fb04-ae55-4b60-b8eb-8b1d02c80941 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:05.804568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6fb04-ae55-4b60-b8eb-8b1d02c80941 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:05.804568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6fb04-ae55-4b60-b8eb-8b1d02c80941 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:05.805873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 515/1001] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:05.811144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1ac80-069b-45a6-8883-a70803840c38 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:05.811588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1ac80-069b-45a6-8883-a70803840c38 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:05.811815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1ac80-069b-45a6-8883-a70803840c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:05.878184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-75d1ac80-069b-45a6-8883-a70803840c38 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:05.879405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-75d1ac80-069b-45a6-8883-a70803840c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:05.988982 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 487/1002] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:06.272281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6fb04-ae55-4b60-b8eb-8b1d02c80941 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:06.273837 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c0d6fb04-ae55-4b60-b8eb-8b1d02c80941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:06.379596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6fb04-ae55-4b60-b8eb-8b1d02c80941 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:06.457804 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6f509-ea60-424e-90d6-b1cf64d101f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:06.458423 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6f509-ea60-424e-90d6-b1cf64d101f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:06.458679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6f509-ea60-424e-90d6-b1cf64d101f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:06.458945 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6f509-ea60-424e-90d6-b1cf64d101f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:06.460573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-05c6f509-ea60-424e-90d6-b1cf64d101f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:06.477506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:06.572001 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6f509-ea60-424e-90d6-b1cf64d101f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:06.573441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6f509-ea60-424e-90d6-b1cf64d101f9 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:06.591583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 516/1003] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:08.866375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef913810-a091-4309-86cc-be1425b3f6af None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:08.866905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef913810-a091-4309-86cc-be1425b3f6af None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:08.866905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef913810-a091-4309-86cc-be1425b3f6af None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:08.868182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 488/1004] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:08.873240 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051c7f-5c5d-4acb-8f05-5c423dc5792a None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:08.873686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051c7f-5c5d-4acb-8f05-5c423dc5792a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:08.873790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051c7f-5c5d-4acb-8f05-5c423dc5792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:08.941175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f0051c7f-5c5d-4acb-8f05-5c423dc5792a None None] MFA Rules not processed for user `cab8f0baebc8466da5e21e19f829447a`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:08.942377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f0051c7f-5c5d-4acb-8f05-5c423dc5792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:09.007232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:09.039080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 517/1005] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:09.322560 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef913810-a091-4309-86cc-be1425b3f6af None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:09.324133 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ef913810-a091-4309-86cc-be1425b3f6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:09.358335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:09.437957 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef913810-a091-4309-86cc-be1425b3f6af None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:09.510973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e07b9-d78a-4aa6-989c-24dd40181194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:09.511491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e07b9-d78a-4aa6-989c-24dd40181194 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:09.511761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e07b9-d78a-4aa6-989c-24dd40181194 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:09.512054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e07b9-d78a-4aa6-989c-24dd40181194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:09.513623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f07e07b9-d78a-4aa6-989c-24dd40181194 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:09.592330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:09.624869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e07b9-d78a-4aa6-989c-24dd40181194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:09.626349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e07b9-d78a-4aa6-989c-24dd40181194 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:09.646245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 489/1006] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:12.032379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34068fc0-0cad-429c-b5ab-328e45c02e06 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:12.032379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34068fc0-0cad-429c-b5ab-328e45c02e06 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:12.033031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34068fc0-0cad-429c-b5ab-328e45c02e06 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:12.033977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 518/1007] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:12.039167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582053b-6e6c-45bb-80f4-99b62997a26b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:12.039571 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582053b-6e6c-45bb-80f4-99b62997a26b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:12.039709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582053b-6e6c-45bb-80f4-99b62997a26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:12.106205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c582053b-6e6c-45bb-80f4-99b62997a26b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:12.107413 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c582053b-6e6c-45bb-80f4-99b62997a26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:12.202005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 490/1008] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:12.481865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34068fc0-0cad-429c-b5ab-328e45c02e06 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:12.483397 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-34068fc0-0cad-429c-b5ab-328e45c02e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:12.586344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34068fc0-0cad-429c-b5ab-328e45c02e06 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:12.657383 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389884c-d3b9-4620-b26a-c1f1e8a30c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:12.657973 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389884c-d3b9-4620-b26a-c1f1e8a30c73 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:12.658243 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389884c-d3b9-4620-b26a-c1f1e8a30c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:12.658503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389884c-d3b9-4620-b26a-c1f1e8a30c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:12.660111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7389884c-d3b9-4620-b26a-c1f1e8a30c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:12.686290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:12.767321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389884c-d3b9-4620-b26a-c1f1e8a30c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:12.768794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389884c-d3b9-4620-b26a-c1f1e8a30c73 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:12.787043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 519/1009] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:14.997607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec927d-be07-44be-9621-a0011347c963 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:14.998144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec927d-be07-44be-9621-a0011347c963 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:14.998144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec927d-be07-44be-9621-a0011347c963 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:14.999439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 491/1010] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:15.005175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420c227-b402-423d-bf80-5667f8af30ea None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:15.005587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420c227-b402-423d-bf80-5667f8af30ea None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:15.005751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420c227-b402-423d-bf80-5667f8af30ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:15.072029 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2420c227-b402-423d-bf80-5667f8af30ea None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:15.073223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2420c227-b402-423d-bf80-5667f8af30ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:15.146512 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:15.169515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 520/1011] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:15.457209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec927d-be07-44be-9621-a0011347c963 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:15.458749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b7ec927d-be07-44be-9621-a0011347c963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:15.498682 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:15.562430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec927d-be07-44be-9621-a0011347c963 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:15.634966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33cce-3c54-450d-ad60-f04a83f12c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:15.635485 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33cce-3c54-450d-ad60-f04a83f12c50 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:15.635762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33cce-3c54-450d-ad60-f04a83f12c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:15.636062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33cce-3c54-450d-ad60-f04a83f12c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:15.637650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-96e33cce-3c54-450d-ad60-f04a83f12c50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:15.724117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:15.749421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33cce-3c54-450d-ad60-f04a83f12c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:15.750932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33cce-3c54-450d-ad60-f04a83f12c50 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:15.769863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 492/1012] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:18.232776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab675d5-c8a9-4e71-afa9-8ca14502c31e None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:18.233258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab675d5-c8a9-4e71-afa9-8ca14502c31e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:18.233367 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab675d5-c8a9-4e71-afa9-8ca14502c31e None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:18.234660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 521/1013] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:18.239847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb93f21-6820-4006-af87-40f7722dc5f1 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:18.240281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb93f21-6820-4006-af87-40f7722dc5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:18.240466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb93f21-6820-4006-af87-40f7722dc5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:18.304987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ebb93f21-6820-4006-af87-40f7722dc5f1 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:18.306162 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ebb93f21-6820-4006-af87-40f7722dc5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:18.401305 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 493/1014] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:18.691164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab675d5-c8a9-4e71-afa9-8ca14502c31e None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:18.692766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1ab675d5-c8a9-4e71-afa9-8ca14502c31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:18.794474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab675d5-c8a9-4e71-afa9-8ca14502c31e None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:18.867625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0abb6-0a77-4e83-a549-c6f560b0bd6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:18.868253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0abb6-0a77-4e83-a549-c6f560b0bd6b service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:18.868524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0abb6-0a77-4e83-a549-c6f560b0bd6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:18.868783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0abb6-0a77-4e83-a549-c6f560b0bd6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:18.870368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cef0abb6-0a77-4e83-a549-c6f560b0bd6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:18.905707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:18.976424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0abb6-0a77-4e83-a549-c6f560b0bd6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:18.977786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0abb6-0a77-4e83-a549-c6f560b0bd6b service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:18.997729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 522/1015] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:23.045325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee43ce7-db01-4342-ac03-3094c59dcb63 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:23.045873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee43ce7-db01-4342-ac03-3094c59dcb63 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:23.045873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee43ce7-db01-4342-ac03-3094c59dcb63 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:23.047169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 494/1016] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:23.053278 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135313f-889c-475b-80fd-bd63e655da1f None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:23.053719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135313f-889c-475b-80fd-bd63e655da1f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:23.053823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135313f-889c-475b-80fd-bd63e655da1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:23.124962 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-d135313f-889c-475b-80fd-bd63e655da1f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:23.126710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d135313f-889c-475b-80fd-bd63e655da1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:23.213845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:23.225746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 523/1017] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:23.511667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee43ce7-db01-4342-ac03-3094c59dcb63 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:23.513297 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-eee43ce7-db01-4342-ac03-3094c59dcb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:23.564850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:23.621487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee43ce7-db01-4342-ac03-3094c59dcb63 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:23.698575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7e82e-4d8a-42f9-96ff-1933a3d3299e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:23.699174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7e82e-4d8a-42f9-96ff-1933a3d3299e service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:23.699449 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7e82e-4d8a-42f9-96ff-1933a3d3299e service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:23.699713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7e82e-4d8a-42f9-96ff-1933a3d3299e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:23.701336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-aba7e82e-4d8a-42f9-96ff-1933a3d3299e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:23.801597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:23.818265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7e82e-4d8a-42f9-96ff-1933a3d3299e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:23.819685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7e82e-4d8a-42f9-96ff-1933a3d3299e service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:23.838603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 495/1018] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:26.662844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aea69-6af5-4eca-b837-efa2121a7df8 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:26.663434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aea69-6af5-4eca-b837-efa2121a7df8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:26.663434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aea69-6af5-4eca-b837-efa2121a7df8 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:26.664728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 524/1019] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:26.669928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d300e-9702-4ac4-89b7-e4b98c5268d9 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:26.670355 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d300e-9702-4ac4-89b7-e4b98c5268d9 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:26.670471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d300e-9702-4ac4-89b7-e4b98c5268d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:26.736874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-951d300e-9702-4ac4-89b7-e4b98c5268d9 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:26.738065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-951d300e-9702-4ac4-89b7-e4b98c5268d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:26.837967 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 496/1020] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:27.121385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730aea69-6af5-4eca-b837-efa2121a7df8 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:27.122920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-730aea69-6af5-4eca-b837-efa2121a7df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:27.231027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730aea69-6af5-4eca-b837-efa2121a7df8 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:27.309813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688824ca-99d7-475b-b33c-3883b319d29f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:27.310432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688824ca-99d7-475b-b33c-3883b319d29f service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:27.310699 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688824ca-99d7-475b-b33c-3883b319d29f service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:27.310959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688824ca-99d7-475b-b33c-3883b319d29f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:27.312559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-688824ca-99d7-475b-b33c-3883b319d29f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:27.356512 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:27.428643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688824ca-99d7-475b-b33c-3883b319d29f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:27.430142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688824ca-99d7-475b-b33c-3883b319d29f service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:27.450361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 525/1021] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:31.605948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2fcfe-69d4-4db1-95a1-c30cb1f0f744 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:31.606461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2fcfe-69d4-4db1-95a1-c30cb1f0f744 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:31.606461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2fcfe-69d4-4db1-95a1-c30cb1f0f744 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:31.607805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 497/1022] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:31.613845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8d5bf-e7e4-49e9-b663-a5661b8a72ac None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:31.614301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8d5bf-e7e4-49e9-b663-a5661b8a72ac None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:31.614404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8d5bf-e7e4-49e9-b663-a5661b8a72ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:31.684647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-cca8d5bf-e7e4-49e9-b663-a5661b8a72ac None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:31.685851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cca8d5bf-e7e4-49e9-b663-a5661b8a72ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:31.786991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:31.789359 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 526/1023] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:32.076488 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2fcfe-69d4-4db1-95a1-c30cb1f0f744 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:32.078013 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-fde2fcfe-69d4-4db1-95a1-c30cb1f0f744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:32.138205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:32.183868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2fcfe-69d4-4db1-95a1-c30cb1f0f744 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:32.263349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec92be-ee1b-44c8-a7ea-8eb9969b7632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:32.264149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec92be-ee1b-44c8-a7ea-8eb9969b7632 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:32.264422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec92be-ee1b-44c8-a7ea-8eb9969b7632 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:32.264683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec92be-ee1b-44c8-a7ea-8eb9969b7632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:32.266494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-73ec92be-ee1b-44c8-a7ea-8eb9969b7632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:32.381197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:32.385059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec92be-ee1b-44c8-a7ea-8eb9969b7632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:32.387984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec92be-ee1b-44c8-a7ea-8eb9969b7632 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:32.406979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 498/1024] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:37.423330 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924ed31-1317-4361-b32e-18af6b37cabb None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:37.424287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924ed31-1317-4361-b32e-18af6b37cabb None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:37.424447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924ed31-1317-4361-b32e-18af6b37cabb None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:37.426403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 527/1025] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:37.432148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb64be5-87ca-4de6-b54b-1db53f0abc8f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:37.432587 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb64be5-87ca-4de6-b54b-1db53f0abc8f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:37.432774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb64be5-87ca-4de6-b54b-1db53f0abc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:37.518006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-fcb64be5-87ca-4de6-b54b-1db53f0abc8f None None] MFA Rules not processed for user `73089bc9f85a405b86c4cd3f8cfa830a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:37.519228 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-fcb64be5-87ca-4de6-b54b-1db53f0abc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:37.692055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 499/1026] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:47:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:41.946729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59afa-43ce-4a8c-b449-8e8360933b6d None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:41.947298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59afa-43ce-4a8c-b449-8e8360933b6d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:41.947298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59afa-43ce-4a8c-b449-8e8360933b6d None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:41.948784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 528/1027] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:41.953529 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ae5fc-f534-4b35-9c13-d01883f6fb12 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:41.954002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ae5fc-f534-4b35-9c13-d01883f6fb12 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:41.954266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ae5fc-f534-4b35-9c13-d01883f6fb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:42.010070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:42.035146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f35ae5fc-f534-4b35-9c13-d01883f6fb12 None None] MFA Rules not processed for user `73089bc9f85a405b86c4cd3f8cfa830a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:42.035671 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f35ae5fc-f534-4b35-9c13-d01883f6fb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:42.152116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 500/1028] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:47:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:43.449204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75943ca-b6f8-45bc-9439-364dc4bd8c79 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:43.449721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75943ca-b6f8-45bc-9439-364dc4bd8c79 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:43.449856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75943ca-b6f8-45bc-9439-364dc4bd8c79 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:43.451289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 529/1029] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:43.456827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03ea8a-38b0-4ba7-8fb8-aabf3956a248 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:43.457231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03ea8a-38b0-4ba7-8fb8-aabf3956a248 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:43.457413 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03ea8a-38b0-4ba7-8fb8-aabf3956a248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:43.535961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-4c03ea8a-38b0-4ba7-8fb8-aabf3956a248 None None] MFA Rules not processed for user `73089bc9f85a405b86c4cd3f8cfa830a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:43.537175 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-4c03ea8a-38b0-4ba7-8fb8-aabf3956a248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:43.578314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:43.643327 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 501/1030] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:47:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:47:47.168465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddc5d6-2566-4513-9d82-f63ff049e30b None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:47.168998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddc5d6-2566-4513-9d82-f63ff049e30b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:47.168998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddc5d6-2566-4513-9d82-f63ff049e30b None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:47.170269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 530/1031] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:47.176529 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43415463-4b55-4ad7-a0e7-d67b099b287c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:47.176970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43415463-4b55-4ad7-a0e7-d67b099b287c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:47.177085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43415463-4b55-4ad7-a0e7-d67b099b287c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:47.248895 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-43415463-4b55-4ad7-a0e7-d67b099b287c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:47.250054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-43415463-4b55-4ad7-a0e7-d67b099b287c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:47.356594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 502/1032] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:47.664941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c844af9-c6fd-4557-b22b-a24e873a13d4 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:47.666033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c844af9-c6fd-4557-b22b-a24e873a13d4 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:47.667035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c844af9-c6fd-4557-b22b-a24e873a13d4 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:47.669033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 531/1033] 10.222.0.27 () {54 vars in 961 bytes} [Fri Oct 14 17:47:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:47.682329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a03094-666a-43aa-9535-a934bee56c4a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:47.682824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a03094-666a-43aa-9535-a934bee56c4a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:47.683302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a03094-666a-43aa-9535-a934bee56c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:47.723085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:47.789974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-72a03094-666a-43aa-9535-a934bee56c4a None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:47.791166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-72a03094-666a-43aa-9535-a934bee56c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:47.929572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 503/1034] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Oct 14 17:47:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:47.937392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c844af9-c6fd-4557-b22b-a24e873a13d4 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:47.939050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2c844af9-c6fd-4557-b22b-a24e873a13d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:48.098399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c844af9-c6fd-4557-b22b-a24e873a13d4 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:48.176760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:48.220624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d7800-7365-4f0d-ab9d-8807b02557b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:48.221994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d7800-7365-4f0d-ab9d-8807b02557b8 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:48.222351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d7800-7365-4f0d-ab9d-8807b02557b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:48.222743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d7800-7365-4f0d-ab9d-8807b02557b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:48.224615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ba6d7800-7365-4f0d-ab9d-8807b02557b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:48.325135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d7800-7365-4f0d-ab9d-8807b02557b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:48.326874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d7800-7365-4f0d-ab9d-8807b02557b8 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:48.352057 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 532/1035] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:50.776764 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb62553-41d8-4a53-b706-a22fb07cab82 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:50.777261 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb62553-41d8-4a53-b706-a22fb07cab82 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:50.777261 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb62553-41d8-4a53-b706-a22fb07cab82 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:50.778903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 504/1036] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:50.787921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee02bb-3d12-4f5d-a070-4520bea71b62 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:50.788326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee02bb-3d12-4f5d-a070-4520bea71b62 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:50.789410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee02bb-3d12-4f5d-a070-4520bea71b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:50.879475 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-49ee02bb-3d12-4f5d-a070-4520bea71b62 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:50.880759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-49ee02bb-3d12-4f5d-a070-4520bea71b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:50.923013 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:50.986491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 533/1037] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:51.278780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb62553-41d8-4a53-b706-a22fb07cab82 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:51.280303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-cbb62553-41d8-4a53-b706-a22fb07cab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:51.355863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:51.465128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb62553-41d8-4a53-b706-a22fb07cab82 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:51.594733 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2dd040-30f4-445a-a175-78ac9b32e0d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:51.595870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dd040-30f4-445a-a175-78ac9b32e0d3 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:51.595870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dd040-30f4-445a-a175-78ac9b32e0d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:51.596767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dd040-30f4-445a-a175-78ac9b32e0d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:51.598530 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0d2dd040-30f4-445a-a175-78ac9b32e0d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:51.622704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:51.696940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2dd040-30f4-445a-a175-78ac9b32e0d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:51.698468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2dd040-30f4-445a-a175-78ac9b32e0d3 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:51.716922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 505/1038] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:54.316836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed62ed-0b6d-45da-9cad-0aa88d854e65 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:54.317399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed62ed-0b6d-45da-9cad-0aa88d854e65 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:54.317689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed62ed-0b6d-45da-9cad-0aa88d854e65 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:54.319166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 534/1039] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:54.324574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227522e4-28d4-442e-8686-592e7ade8261 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:54.325031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227522e4-28d4-442e-8686-592e7ade8261 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:54.325126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227522e4-28d4-442e-8686-592e7ade8261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:54.407148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-227522e4-28d4-442e-8686-592e7ade8261 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:54.409017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-227522e4-28d4-442e-8686-592e7ade8261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:54.520666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:54.528288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 506/1040] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:54.811271 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5926df2-2549-4c63-a42f-3d17490767ab None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:54.811705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5926df2-2549-4c63-a42f-3d17490767ab None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:54.812120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5926df2-2549-4c63-a42f-3d17490767ab None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:54.813639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 535/1041] 10.222.0.27 () {54 vars in 961 bytes} [Fri Oct 14 17:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:54.819567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8f994-d9b5-4acc-ab6b-13a40da0475e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:54.819841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8f994-d9b5-4acc-ab6b-13a40da0475e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:54.820138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8f994-d9b5-4acc-ab6b-13a40da0475e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:54.898402 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-bbb8f994-d9b5-4acc-ab6b-13a40da0475e None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:54.899591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-bbb8f994-d9b5-4acc-ab6b-13a40da0475e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:55.020193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 507/1042] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Oct 14 17:47:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:55.027923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5926df2-2549-4c63-a42f-3d17490767ab None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:55.029541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c5926df2-2549-4c63-a42f-3d17490767ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:55.182691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5926df2-2549-4c63-a42f-3d17490767ab None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:55.204061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:55.310384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d08bc6-0831-4e4d-9d15-6a2f7a0a214b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:55.311010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d08bc6-0831-4e4d-9d15-6a2f7a0a214b service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:55.311268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d08bc6-0831-4e4d-9d15-6a2f7a0a214b service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:55.311533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d08bc6-0831-4e4d-9d15-6a2f7a0a214b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:55.313138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b8d08bc6-0831-4e4d-9d15-6a2f7a0a214b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:55.418957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d08bc6-0831-4e4d-9d15-6a2f7a0a214b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:55.420631 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d08bc6-0831-4e4d-9d15-6a2f7a0a214b service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:55.441928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 536/1043] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:55.517929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b319edc-01fb-4a14-a9bc-664cbf9a00b3 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:55.519207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b319edc-01fb-4a14-a9bc-664cbf9a00b3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:55.519207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b319edc-01fb-4a14-a9bc-664cbf9a00b3 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:55.520771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 508/1044] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:55.526110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e188cbb-574c-4d19-b4dd-674f7ef1e468 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:55.526463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e188cbb-574c-4d19-b4dd-674f7ef1e468 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:55.526662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e188cbb-574c-4d19-b4dd-674f7ef1e468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:55.566783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:55.611782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4e188cbb-574c-4d19-b4dd-674f7ef1e468 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:55.613067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4e188cbb-574c-4d19-b4dd-674f7ef1e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:55.734186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 537/1045] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:56.090469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388d0a2-923b-4f1e-ba49-a87d3218505b None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:56.090469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388d0a2-923b-4f1e-ba49-a87d3218505b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:56.090469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388d0a2-923b-4f1e-ba49-a87d3218505b None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:56.093210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 509/1046] 10.222.0.27 () {54 vars in 963 bytes} [Fri Oct 14 17:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:56.096109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62747f4-663a-4520-8b9c-d6872db20b40 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:56.096930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62747f4-663a-4520-8b9c-d6872db20b40 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:56.097116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62747f4-663a-4520-8b9c-d6872db20b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:56.177439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-b62747f4-663a-4520-8b9c-d6872db20b40 None None] MFA Rules not processed for user `8f51ee9e45fb4dda8d5b9740aa0ac828`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:56.178828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b62747f4-663a-4520-8b9c-d6872db20b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:56.233130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:56.266620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 538/1047] 10.222.0.27 () {58 vars in 1060 bytes} [Fri Oct 14 17:47:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:56.274723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388d0a2-923b-4f1e-ba49-a87d3218505b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:56.276272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6388d0a2-923b-4f1e-ba49-a87d3218505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:56.330309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:56.396193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388d0a2-923b-4f1e-ba49-a87d3218505b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:56.478728 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7633f1-7869-4127-a511-e449ee456203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:56.479532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7633f1-7869-4127-a511-e449ee456203 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:56.479731 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7633f1-7869-4127-a511-e449ee456203 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:56.480127 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7633f1-7869-4127-a511-e449ee456203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:56.481642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1a7633f1-7869-4127-a511-e449ee456203 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:56.601274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:56.617058 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7633f1-7869-4127-a511-e449ee456203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:56.618621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7633f1-7869-4127-a511-e449ee456203 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:56.638205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 510/1048] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:47:57.712095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dc328-c2c7-4f4e-a02b-ff076bc25fb3 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:57.712627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dc328-c2c7-4f4e-a02b-ff076bc25fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:57.712627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dc328-c2c7-4f4e-a02b-ff076bc25fb3 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:57.713950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 539/1049] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:47:57.719221 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb430e-697f-4b11-a980-d20a643f9b4d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:57.719727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb430e-697f-4b11-a980-d20a643f9b4d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:57.719826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb430e-697f-4b11-a980-d20a643f9b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:57.786202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-dddb430e-697f-4b11-a980-d20a643f9b4d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:47:57.787405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-dddb430e-697f-4b11-a980-d20a643f9b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:57.884699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 511/1050] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:47:58.179807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dc328-c2c7-4f4e-a02b-ff076bc25fb3 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:47:58.181624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ae1dc328-c2c7-4f4e-a02b-ff076bc25fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:58.338610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dc328-c2c7-4f4e-a02b-ff076bc25fb3 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:47:58.392559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:47:58.459403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388384d-7b74-41c8-a4da-415b6f4ed9ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:47:58.460163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388384d-7b74-41c8-a4da-415b6f4ed9ac service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:47:58.460443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388384d-7b74-41c8-a4da-415b6f4ed9ac service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:47:58.460714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388384d-7b74-41c8-a4da-415b6f4ed9ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:47:58.462321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a388384d-7b74-41c8-a4da-415b6f4ed9ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:47:58.556574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388384d-7b74-41c8-a4da-415b6f4ed9ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:47:58.558189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388384d-7b74-41c8-a4da-415b6f4ed9ac service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:47:58.576852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 540/1051] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:00.850809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471d5f-b0fd-4725-a909-7afdd0b84913 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:00.851342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471d5f-b0fd-4725-a909-7afdd0b84913 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:00.851342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471d5f-b0fd-4725-a909-7afdd0b84913 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:00.852729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 512/1052] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:00.858065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61249f6-04ff-4034-9ce7-0b1ad12a1f55 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:00.858464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61249f6-04ff-4034-9ce7-0b1ad12a1f55 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:00.858631 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61249f6-04ff-4034-9ce7-0b1ad12a1f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:00.928744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f61249f6-04ff-4034-9ce7-0b1ad12a1f55 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:00.929963 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f61249f6-04ff-4034-9ce7-0b1ad12a1f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:00.937857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:01.036808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 541/1053] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:01.331577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e471d5f-b0fd-4725-a909-7afdd0b84913 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:01.333366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7e471d5f-b0fd-4725-a909-7afdd0b84913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:01.402089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:01.491896 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e471d5f-b0fd-4725-a909-7afdd0b84913 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:01.609558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a2df0-777f-4125-a57c-dd17a0118c65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:01.610193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a2df0-777f-4125-a57c-dd17a0118c65 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:01.610467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a2df0-777f-4125-a57c-dd17a0118c65 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:01.610730 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a2df0-777f-4125-a57c-dd17a0118c65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:01.612871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ce5a2df0-777f-4125-a57c-dd17a0118c65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:01.633108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:01.709615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a2df0-777f-4125-a57c-dd17a0118c65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:01.711200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a2df0-777f-4125-a57c-dd17a0118c65 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:01.731716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 513/1054] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:48:03.992319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6852c-06dd-4b83-bec8-bf4cc08ec351 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:03.992763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6852c-06dd-4b83-bec8-bf4cc08ec351 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:03.992929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6852c-06dd-4b83-bec8-bf4cc08ec351 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:03.994494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 542/1055] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:04.001582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e61c9-c0b5-484e-acfe-ade22dc371bb None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:04.001993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e61c9-c0b5-484e-acfe-ade22dc371bb None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:04.002314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e61c9-c0b5-484e-acfe-ade22dc371bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:04.081706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-b43e61c9-c0b5-484e-acfe-ade22dc371bb None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:04.082865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b43e61c9-c0b5-484e-acfe-ade22dc371bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:04.188027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:04.194732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 514/1056] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:04.491518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6852c-06dd-4b83-bec8-bf4cc08ec351 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:04.493023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-59a6852c-06dd-4b83-bec8-bf4cc08ec351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:04.678157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:04.693865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6852c-06dd-4b83-bec8-bf4cc08ec351 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:04.828802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36219e-888a-49bf-8dad-d93055bcaffa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:04.829640 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36219e-888a-49bf-8dad-d93055bcaffa service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:04.829984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36219e-888a-49bf-8dad-d93055bcaffa service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:04.830255 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36219e-888a-49bf-8dad-d93055bcaffa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:04.832433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-db36219e-888a-49bf-8dad-d93055bcaffa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:04.930412 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36219e-888a-49bf-8dad-d93055bcaffa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:04.932055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36219e-888a-49bf-8dad-d93055bcaffa service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:04.941979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:04.950782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 543/1057] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:07.209349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796b22-0b1f-43ba-8b5f-6410327c9c98 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:07.209880 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796b22-0b1f-43ba-8b5f-6410327c9c98 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:07.210007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796b22-0b1f-43ba-8b5f-6410327c9c98 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:07.211955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 515/1058] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:48:07.218130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755a450-2dd2-4e35-8255-af58be03ebc6 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:07.218560 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755a450-2dd2-4e35-8255-af58be03ebc6 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:07.218643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755a450-2dd2-4e35-8255-af58be03ebc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:07.286682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7755a450-2dd2-4e35-8255-af58be03ebc6 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:07.287914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7755a450-2dd2-4e35-8255-af58be03ebc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:07.397238 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 544/1059] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:48:07.689058 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01796b22-0b1f-43ba-8b5f-6410327c9c98 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:07.691453 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-01796b22-0b1f-43ba-8b5f-6410327c9c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:07.854727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01796b22-0b1f-43ba-8b5f-6410327c9c98 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:07.927172 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:07.975326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aada93-7ef9-4c0d-936f-0b9f4de00ce3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:07.975964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aada93-7ef9-4c0d-936f-0b9f4de00ce3 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:07.976246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aada93-7ef9-4c0d-936f-0b9f4de00ce3 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:07.976526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aada93-7ef9-4c0d-936f-0b9f4de00ce3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:07.978101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-19aada93-7ef9-4c0d-936f-0b9f4de00ce3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:08.074807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aada93-7ef9-4c0d-936f-0b9f4de00ce3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:08.076431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aada93-7ef9-4c0d-936f-0b9f4de00ce3 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:08.095965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 516/1060] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:10.307635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba139b7-1be4-4eb7-b512-7af22f790eeb None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:10.308256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba139b7-1be4-4eb7-b512-7af22f790eeb None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:10.308513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba139b7-1be4-4eb7-b512-7af22f790eeb None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:10.310194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 545/1061] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:10.316209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c09022-29dd-494b-b0bc-2178810ab049 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:10.317002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c09022-29dd-494b-b0bc-2178810ab049 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:10.317002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c09022-29dd-494b-b0bc-2178810ab049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:10.385098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-89c09022-29dd-494b-b0bc-2178810ab049 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:10.386314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-89c09022-29dd-494b-b0bc-2178810ab049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:10.422263 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:10.491862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 517/1062] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:10.854434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba139b7-1be4-4eb7-b512-7af22f790eeb None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:10.856045 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cba139b7-1be4-4eb7-b512-7af22f790eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:10.934162 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:11.018710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba139b7-1be4-4eb7-b512-7af22f790eeb None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:11.138649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844ee7d-5f16-4b73-b721-f7ae7c3473c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:11.139171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844ee7d-5f16-4b73-b721-f7ae7c3473c8 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:11.139439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844ee7d-5f16-4b73-b721-f7ae7c3473c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:11.139702 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844ee7d-5f16-4b73-b721-f7ae7c3473c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:11.141302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4844ee7d-5f16-4b73-b721-f7ae7c3473c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:11.181717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:11.233183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844ee7d-5f16-4b73-b721-f7ae7c3473c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:11.234784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844ee7d-5f16-4b73-b721-f7ae7c3473c8 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:11.252956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 546/1063] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:13.501937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdef03-ddf6-4955-9047-0e120adf8862 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:13.502441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdef03-ddf6-4955-9047-0e120adf8862 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:13.502441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdef03-ddf6-4955-9047-0e120adf8862 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:13.504676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 518/1064] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:48:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:13.510011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef6730-cf16-467b-9a84-3c40a588106e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:13.510441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef6730-cf16-467b-9a84-3c40a588106e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:13.510564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef6730-cf16-467b-9a84-3c40a588106e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:13.582511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9eef6730-cf16-467b-9a84-3c40a588106e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:13.584260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9eef6730-cf16-467b-9a84-3c40a588106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:13.700213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 547/1065] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:13.990645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdef03-ddf6-4955-9047-0e120adf8862 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:13.992345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a1fdef03-ddf6-4955-9047-0e120adf8862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:14.141186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdef03-ddf6-4955-9047-0e120adf8862 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:14.159120 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:14.262116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9b7d5-4c07-4c5c-95c6-ec25e6db502b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:14.262763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9b7d5-4c07-4c5c-95c6-ec25e6db502b service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:14.263036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9b7d5-4c07-4c5c-95c6-ec25e6db502b service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:14.263298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9b7d5-4c07-4c5c-95c6-ec25e6db502b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:14.264927 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-98c9b7d5-4c07-4c5c-95c6-ec25e6db502b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:14.359917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9b7d5-4c07-4c5c-95c6-ec25e6db502b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:14.361515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9b7d5-4c07-4c5c-95c6-ec25e6db502b service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:14.380027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 519/1066] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:15.624674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d66bc-f5c1-4009-b268-e34f8e0fb536 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:15.625220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d66bc-f5c1-4009-b268-e34f8e0fb536 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:15.625220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d66bc-f5c1-4009-b268-e34f8e0fb536 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:15.626452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 548/1067] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:15.632253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14609c-2e48-4f24-b81d-3a8b1cb69c1d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:15.632681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14609c-2e48-4f24-b81d-3a8b1cb69c1d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:15.632796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14609c-2e48-4f24-b81d-3a8b1cb69c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:15.658988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:15.701470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ca14609c-2e48-4f24-b81d-3a8b1cb69c1d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:15.702710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ca14609c-2e48-4f24-b81d-3a8b1cb69c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:15.798836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 520/1068] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:16.101805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d335784-d62d-4ad7-b9f8-c6ad1b4bfdf7 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:16.102083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d335784-d62d-4ad7-b9f8-c6ad1b4bfdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:16.102350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d335784-d62d-4ad7-b9f8-c6ad1b4bfdf7 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:16.103690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 549/1069] 10.222.0.27 () {54 vars in 962 bytes} [Fri Oct 14 17:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:16.109377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f4a2a-415e-44b6-8743-b7daa75f0255 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:16.109689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f4a2a-415e-44b6-8743-b7daa75f0255 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:16.109961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f4a2a-415e-44b6-8743-b7daa75f0255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:16.189164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-012f4a2a-415e-44b6-8743-b7daa75f0255 None None] MFA Rules not processed for user `ae3566d85c9c4c559ba672269ab38f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:16.190345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-012f4a2a-415e-44b6-8743-b7daa75f0255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:16.232561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:16.272912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 521/1070] 10.222.0.27 () {58 vars in 1059 bytes} [Fri Oct 14 17:48:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:16.282216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d335784-d62d-4ad7-b9f8-c6ad1b4bfdf7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:16.283989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8d335784-d62d-4ad7-b9f8-c6ad1b4bfdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:16.297765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:16.392256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d335784-d62d-4ad7-b9f8-c6ad1b4bfdf7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:16.470042 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b3b13-b760-450b-9773-675beb0fd9b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3566d85c9c4c559ba672269ab38f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:16.470659 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b3b13-b760-450b-9773-675beb0fd9b1 service glance] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:16.470929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b3b13-b760-450b-9773-675beb0fd9b1 service glance] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:16.471210 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b3b13-b760-450b-9773-675beb0fd9b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:16.473951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-de0b3b13-b760-450b-9773-675beb0fd9b1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:16.547082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:16.588131 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74239-04fe-4057-a8c3-16cbd1ee174d None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:16.588512 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74239-04fe-4057-a8c3-16cbd1ee174d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:16.588856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74239-04fe-4057-a8c3-16cbd1ee174d None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:16.590308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 522/1071] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:48:16.591267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b3b13-b760-450b-9773-675beb0fd9b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:16.592821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b3b13-b760-450b-9773-675beb0fd9b1 service glance] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:16.595402 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdcb8f-7367-4564-bebc-6473f127f764 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:16.595679 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdcb8f-7367-4564-bebc-6473f127f764 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:16.596002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdcb8f-7367-4564-bebc-6473f127f764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:16.611973 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 550/1072] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:16.662619 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ebfdcb8f-7367-4564-bebc-6473f127f764 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:16.663800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ebfdcb8f-7367-4564-bebc-6473f127f764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:16.758429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 523/1073] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:48:17.058974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b3b13-b760-450b-9773-675beb0fd9b1 service glance] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:17.060317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-de0b3b13-b760-450b-9773-675beb0fd9b1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:17.160606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b3b13-b760-450b-9773-675beb0fd9b1 service glance] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:17.235293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:17.285991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988855a-9752-4fd6-a51b-b0b25d72b0b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:17.286511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988855a-9752-4fd6-a51b-b0b25d72b0b2 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:17.286785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988855a-9752-4fd6-a51b-b0b25d72b0b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:17.287053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988855a-9752-4fd6-a51b-b0b25d72b0b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:17.288672 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a988855a-9752-4fd6-a51b-b0b25d72b0b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:17.368820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b532b5b-def2-4ba2-904f-7cfe41b2935e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:17.369110 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b532b5b-def2-4ba2-904f-7cfe41b2935e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:17.369391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b532b5b-def2-4ba2-904f-7cfe41b2935e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:17.389525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988855a-9752-4fd6-a51b-b0b25d72b0b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:17.391761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988855a-9752-4fd6-a51b-b0b25d72b0b2 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:17.412299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 551/1074] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:17.431660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:17.442749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6b532b5b-def2-4ba2-904f-7cfe41b2935e None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:17.444036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6b532b5b-def2-4ba2-904f-7cfe41b2935e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:17.544993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 524/1075] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:48:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:48:17.591868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdda2b0-87b9-43e1-ac8e-f268d5b8a398 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:17.592190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdda2b0-87b9-43e1-ac8e-f268d5b8a398 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:17.592464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdda2b0-87b9-43e1-ac8e-f268d5b8a398 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:17.593887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 552/1076] 10.222.0.27 () {54 vars in 943 bytes} [Fri Oct 14 17:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:17.601049 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b4c88-832c-48b6-ba06-b3089f30045f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:17.601335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b4c88-832c-48b6-ba06-b3089f30045f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:17.601606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b4c88-832c-48b6-ba06-b3089f30045f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:17.685831 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-fc6b4c88-832c-48b6-ba06-b3089f30045f None None] MFA Rules not processed for user `9ec9c3fcf7964640a5eeabc01cd97617`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:17.687296 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-fc6b4c88-832c-48b6-ba06-b3089f30045f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:17.763458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:17.800977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 525/1077] 10.222.0.27 () {58 vars in 1040 bytes} [Fri Oct 14 17:48:17 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:17.810466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdda2b0-87b9-43e1-ac8e-f268d5b8a398 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:17.812011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9fdda2b0-87b9-43e1-ac8e-f268d5b8a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:17.893305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:17.951783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdda2b0-87b9-43e1-ac8e-f268d5b8a398 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:18.058003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517e059-f524-44c5-bb21-c5d1518e861c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:18.058533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517e059-f524-44c5-bb21-c5d1518e861c service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:18.058810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517e059-f524-44c5-bb21-c5d1518e861c service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:18.059081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517e059-f524-44c5-bb21-c5d1518e861c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:18.060846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4517e059-f524-44c5-bb21-c5d1518e861c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:18.126552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:18.147579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517e059-f524-44c5-bb21-c5d1518e861c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:18.149290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517e059-f524-44c5-bb21-c5d1518e861c service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:18.157378 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 553/1078] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:48:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:48:19.665569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f1890-1eb4-4f5d-b266-0ec1c6fc5b58 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:19.666206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f1890-1eb4-4f5d-b266-0ec1c6fc5b58 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:19.666206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f1890-1eb4-4f5d-b266-0ec1c6fc5b58 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:19.667363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 526/1079] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:19.672776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4da772-62bc-4dca-9b86-b6d2a1963c44 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:19.673288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4da772-62bc-4dca-9b86-b6d2a1963c44 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:19.674188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4da772-62bc-4dca-9b86-b6d2a1963c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:19.747035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-db4da772-62bc-4dca-9b86-b6d2a1963c44 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:19.748378 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-db4da772-62bc-4dca-9b86-b6d2a1963c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:19.854768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 554/1080] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:20.145749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f1890-1eb4-4f5d-b266-0ec1c6fc5b58 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:20.147520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-384f1890-1eb4-4f5d-b266-0ec1c6fc5b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:20.315648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f1890-1eb4-4f5d-b266-0ec1c6fc5b58 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:20.377509 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:20.456369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4359a-6625-4ab6-a242-d63775a185f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:20.457078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4359a-6625-4ab6-a242-d63775a185f0 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:20.457501 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4359a-6625-4ab6-a242-d63775a185f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:20.457622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4359a-6625-4ab6-a242-d63775a185f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:20.459240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2cf4359a-6625-4ab6-a242-d63775a185f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:20.581116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4359a-6625-4ab6-a242-d63775a185f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:20.582879 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4359a-6625-4ab6-a242-d63775a185f0 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:20.602133 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 527/1081] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:48:22.811047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cb445-a70c-4e42-8667-5c32ca26e0b7 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:22.811605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cb445-a70c-4e42-8667-5c32ca26e0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:22.811605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cb445-a70c-4e42-8667-5c32ca26e0b7 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:22.812940 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 555/1082] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:22.818998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c67377-068f-44fb-b211-d78351a08d4b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:22.820026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c67377-068f-44fb-b211-d78351a08d4b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:22.820362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c67377-068f-44fb-b211-d78351a08d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:22.892659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c1c67377-068f-44fb-b211-d78351a08d4b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:22.893915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c1c67377-068f-44fb-b211-d78351a08d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:22.897087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:23.008067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 528/1083] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:23.301981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cb445-a70c-4e42-8667-5c32ca26e0b7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:23.303556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a82cb445-a70c-4e42-8667-5c32ca26e0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:23.455374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:23.601205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cb445-a70c-4e42-8667-5c32ca26e0b7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:23.728666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398413da-2628-4a1d-a5c9-d1f2f70cdca4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:23.729275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398413da-2628-4a1d-a5c9-d1f2f70cdca4 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:23.729551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398413da-2628-4a1d-a5c9-d1f2f70cdca4 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:23.729817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398413da-2628-4a1d-a5c9-d1f2f70cdca4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:23.731413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-398413da-2628-4a1d-a5c9-d1f2f70cdca4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:23.750954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:23.837725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398413da-2628-4a1d-a5c9-d1f2f70cdca4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:23.839326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398413da-2628-4a1d-a5c9-d1f2f70cdca4 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:23.859041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 556/1084] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:33.001250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=41 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:33.856275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=37 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:40.946607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c30827-d3ff-42cf-a85e-244acf99704e None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:40.947182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c30827-d3ff-42cf-a85e-244acf99704e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:40.947182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c30827-d3ff-42cf-a85e-244acf99704e None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:40.948510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 529/1085] 10.222.0.27 () {54 vars in 918 bytes} [Fri Oct 14 17:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:40.953624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a6705-d3f5-4134-8b96-5ef874612751 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:40.954060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a6705-d3f5-4134-8b96-5ef874612751 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:40.954251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a6705-d3f5-4134-8b96-5ef874612751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:41.030163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ae8a6705-d3f5-4134-8b96-5ef874612751 None None] MFA Rules not processed for user `73089bc9f85a405b86c4cd3f8cfa830a`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:41.031386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ae8a6705-d3f5-4134-8b96-5ef874612751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:41.136910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 557/1086] 10.222.0.27 () {58 vars in 1015 bytes} [Fri Oct 14 17:48:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:41.151236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2f122-4c26-4f45-a15a-0fe3b530deff None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:41.151511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2f122-4c26-4f45-a15a-0fe3b530deff None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:41.151777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2f122-4c26-4f45-a15a-0fe3b530deff None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:41.153107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 530/1087] 10.222.0.27 () {54 vars in 918 bytes} [Fri Oct 14 17:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:41.157910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6565ed-726f-4c4a-94c8-474f19e170cf None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:41.158196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6565ed-726f-4c4a-94c8-474f19e170cf None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:41.158472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6565ed-726f-4c4a-94c8-474f19e170cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:41.230585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:41.233651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ad6565ed-726f-4c4a-94c8-474f19e170cf None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:41.235398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ad6565ed-726f-4c4a-94c8-474f19e170cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:41.350084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 558/1088] 10.222.0.27 () {58 vars in 1015 bytes} [Fri Oct 14 17:48:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:43.917657 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bef863-6813-46be-83e3-3d48eef0e57a None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:43.918231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bef863-6813-46be-83e3-3d48eef0e57a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:43.918231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bef863-6813-46be-83e3-3d48eef0e57a None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:43.919563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 531/1089] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:43.924799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae17cd2-2064-4ba9-b146-891f7794f38e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:43.925222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae17cd2-2064-4ba9-b146-891f7794f38e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:43.925334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae17cd2-2064-4ba9-b146-891f7794f38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:43.993739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1ae17cd2-2064-4ba9-b146-891f7794f38e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:43.994925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1ae17cd2-2064-4ba9-b146-891f7794f38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:44.049483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:44.093222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 559/1090] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:48:44.382746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bef863-6813-46be-83e3-3d48eef0e57a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:44.384445 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d7bef863-6813-46be-83e3-3d48eef0e57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:44.527069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bef863-6813-46be-83e3-3d48eef0e57a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:44.599622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:44.639085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbe05e-9afc-421e-bc5c-4a9abeb0dbd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:44.639680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe05e-9afc-421e-bc5c-4a9abeb0dbd3 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:44.639987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe05e-9afc-421e-bc5c-4a9abeb0dbd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:44.640256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe05e-9afc-421e-bc5c-4a9abeb0dbd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:44.641848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-73cbe05e-9afc-421e-bc5c-4a9abeb0dbd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:44.731189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cbe05e-9afc-421e-bc5c-4a9abeb0dbd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:44.732805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cbe05e-9afc-421e-bc5c-4a9abeb0dbd3 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:44.750491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 532/1091] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:45.888859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f85eab-2fd4-4058-a924-83880d4dc885 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:45.889449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f85eab-2fd4-4058-a924-83880d4dc885 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:45.889449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f85eab-2fd4-4058-a924-83880d4dc885 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:45.890782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 560/1092] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:45.897314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479e68a-1984-46d1-8897-11282350ce78 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:45.897755 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479e68a-1984-46d1-8897-11282350ce78 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:45.897860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479e68a-1984-46d1-8897-11282350ce78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:45.974936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a479e68a-1984-46d1-8897-11282350ce78 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:45.976171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a479e68a-1984-46d1-8897-11282350ce78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:46.003032 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:46.074023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 533/1093] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:46.228291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f85eab-2fd4-4058-a924-83880d4dc885 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:46.229793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-89f85eab-2fd4-4058-a924-83880d4dc885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:46.376428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f85eab-2fd4-4058-a924-83880d4dc885 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:46.417670 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:46.491838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a4e94-5b73-40a9-961a-6fbfc4770e19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:46.492385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a4e94-5b73-40a9-961a-6fbfc4770e19 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:46.492653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a4e94-5b73-40a9-961a-6fbfc4770e19 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:46.492916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a4e94-5b73-40a9-961a-6fbfc4770e19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:46.494483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c33a4e94-5b73-40a9-961a-6fbfc4770e19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:46.584284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a4e94-5b73-40a9-961a-6fbfc4770e19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:46.585859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a4e94-5b73-40a9-961a-6fbfc4770e19 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:46.603577 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 561/1094] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:47.734501 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc88b7-538b-4146-a4af-051126144f67 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:47.735042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc88b7-538b-4146-a4af-051126144f67 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:47.735042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc88b7-538b-4146-a4af-051126144f67 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:47.736421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 534/1095] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:47.743408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcd9b4-fd93-46be-9ecf-2aa4102f15c2 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:47.743857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcd9b4-fd93-46be-9ecf-2aa4102f15c2 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:47.743990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcd9b4-fd93-46be-9ecf-2aa4102f15c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:47.797702 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:47.821083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c6bcd9b4-fd93-46be-9ecf-2aa4102f15c2 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:47.822293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c6bcd9b4-fd93-46be-9ecf-2aa4102f15c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:47.922412 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 562/1096] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:48.083844 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc88b7-538b-4146-a4af-051126144f67 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:48.085372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b5bc88b7-538b-4146-a4af-051126144f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:48.230170 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc88b7-538b-4146-a4af-051126144f67 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:48.243502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:48.344520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b829d3e-2ae5-48fa-af2b-72a0511ee3ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:48.345055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b829d3e-2ae5-48fa-af2b-72a0511ee3ec service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:48.345323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b829d3e-2ae5-48fa-af2b-72a0511ee3ec service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:48.345583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b829d3e-2ae5-48fa-af2b-72a0511ee3ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:48.347159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8b829d3e-2ae5-48fa-af2b-72a0511ee3ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:48.438259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b829d3e-2ae5-48fa-af2b-72a0511ee3ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:48.439834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b829d3e-2ae5-48fa-af2b-72a0511ee3ec service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:48.457385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 535/1097] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:49.570928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7fa27-f509-4bea-ad2d-119a62da70d9 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:49.571486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7fa27-f509-4bea-ad2d-119a62da70d9 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:49.571486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7fa27-f509-4bea-ad2d-119a62da70d9 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:49.572824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 563/1098] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:49.579167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92da71-2e66-4308-b2aa-14aaab41c7a7 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:49.579611 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92da71-2e66-4308-b2aa-14aaab41c7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:49.579730 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92da71-2e66-4308-b2aa-14aaab41c7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:49.600208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:49.657704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ac92da71-2e66-4308-b2aa-14aaab41c7a7 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:49.658898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ac92da71-2e66-4308-b2aa-14aaab41c7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:49.754753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 536/1099] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:49.903766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7fa27-f509-4bea-ad2d-119a62da70d9 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:49.905299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-97a7fa27-f509-4bea-ad2d-119a62da70d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:50.014765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:50.052989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7fa27-f509-4bea-ad2d-119a62da70d9 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:50.163414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64472884-c047-4804-aae9-d85d6a4e8e2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:50.164106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64472884-c047-4804-aae9-d85d6a4e8e2d service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:50.164384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64472884-c047-4804-aae9-d85d6a4e8e2d service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:50.164649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64472884-c047-4804-aae9-d85d6a4e8e2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:50.166239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-64472884-c047-4804-aae9-d85d6a4e8e2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:50.241229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:50.259199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64472884-c047-4804-aae9-d85d6a4e8e2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:50.260902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64472884-c047-4804-aae9-d85d6a4e8e2d service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:50.279398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 564/1100] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:51.389464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe755e-0479-447f-9be7-f1e792fdbaa3 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:51.389960 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe755e-0479-447f-9be7-f1e792fdbaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:51.390066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe755e-0479-447f-9be7-f1e792fdbaa3 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:51.391378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 537/1101] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:51.397902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50721bbf-5dc0-42cd-a1f6-1d72786dc2b5 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:51.398335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50721bbf-5dc0-42cd-a1f6-1d72786dc2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:51.398467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50721bbf-5dc0-42cd-a1f6-1d72786dc2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:51.474403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-50721bbf-5dc0-42cd-a1f6-1d72786dc2b5 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:51.475593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-50721bbf-5dc0-42cd-a1f6-1d72786dc2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:51.571388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 565/1102] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:51.722072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe755e-0479-447f-9be7-f1e792fdbaa3 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:51.723519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-04fe755e-0479-447f-9be7-f1e792fdbaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:51.806675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:51.867472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe755e-0479-447f-9be7-f1e792fdbaa3 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:51.979148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14362828-409c-49a0-89e8-b3bc04b81d97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:51.979668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14362828-409c-49a0-89e8-b3bc04b81d97 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:51.979968 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14362828-409c-49a0-89e8-b3bc04b81d97 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:51.980237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14362828-409c-49a0-89e8-b3bc04b81d97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:51.981828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-14362828-409c-49a0-89e8-b3bc04b81d97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:52.031014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:52.074419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14362828-409c-49a0-89e8-b3bc04b81d97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:52.076049 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14362828-409c-49a0-89e8-b3bc04b81d97 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:52.094398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 538/1103] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:53.151395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67720e-6d31-470f-89f7-f256c6e5f4b6 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:53.151977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67720e-6d31-470f-89f7-f256c6e5f4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:53.151977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67720e-6d31-470f-89f7-f256c6e5f4b6 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:53.153290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 566/1104] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:53.159457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d99c9-3e27-4835-b5c2-9963348d81bf None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:53.159958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d99c9-3e27-4835-b5c2-9963348d81bf None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:53.160057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d99c9-3e27-4835-b5c2-9963348d81bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:53.237111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-5f9d99c9-3e27-4835-b5c2-9963348d81bf None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:53.238302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5f9d99c9-3e27-4835-b5c2-9963348d81bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:53.337164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 539/1105] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:53.491206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67720e-6d31-470f-89f7-f256c6e5f4b6 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:53.492728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4e67720e-6d31-470f-89f7-f256c6e5f4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:53.542029 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:53.637709 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67720e-6d31-470f-89f7-f256c6e5f4b6 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:53.747752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf2ad6-9956-4799-9411-88661b8c5c92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:53.748293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf2ad6-9956-4799-9411-88661b8c5c92 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:53.748564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf2ad6-9956-4799-9411-88661b8c5c92 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:53.748831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf2ad6-9956-4799-9411-88661b8c5c92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:53.750402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8fcf2ad6-9956-4799-9411-88661b8c5c92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:53.757340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:53.838634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf2ad6-9956-4799-9411-88661b8c5c92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:53.840246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf2ad6-9956-4799-9411-88661b8c5c92 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:53.858088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 567/1106] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:54.917583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d13f01-ce5e-4671-8bd0-636c35d3ed01 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:54.918251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d13f01-ce5e-4671-8bd0-636c35d3ed01 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:54.918251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d13f01-ce5e-4671-8bd0-636c35d3ed01 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:54.919477 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 540/1107] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:54.925864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993d829-0172-4595-8288-0b3d25ed1b9e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:54.926315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993d829-0172-4595-8288-0b3d25ed1b9e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:54.926421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993d829-0172-4595-8288-0b3d25ed1b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:55.004535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-b993d829-0172-4595-8288-0b3d25ed1b9e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:55.006160 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b993d829-0172-4595-8288-0b3d25ed1b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:55.077756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:55.107452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 568/1108] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:55.257888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d13f01-ce5e-4671-8bd0-636c35d3ed01 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:55.259314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e9d13f01-ce5e-4671-8bd0-636c35d3ed01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:55.277860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:55.407708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d13f01-ce5e-4671-8bd0-636c35d3ed01 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:55.494171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:55.519095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac63182-b0fd-4dcb-a3e2-79ad729fd25f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:55.519629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac63182-b0fd-4dcb-a3e2-79ad729fd25f service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:55.519925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac63182-b0fd-4dcb-a3e2-79ad729fd25f service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:55.520193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac63182-b0fd-4dcb-a3e2-79ad729fd25f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:55.521978 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8ac63182-b0fd-4dcb-a3e2-79ad729fd25f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:55.612645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac63182-b0fd-4dcb-a3e2-79ad729fd25f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:55.614219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac63182-b0fd-4dcb-a3e2-79ad729fd25f service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:55.631847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 541/1109] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:48:56.689192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41987e2-cfc5-43c6-b0e4-e27470b4738b None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:56.689740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41987e2-cfc5-43c6-b0e4-e27470b4738b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:56.689856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41987e2-cfc5-43c6-b0e4-e27470b4738b None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:56.691133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 569/1110] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:56.697448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1178c0a-2720-4600-805e-893273c2c1c9 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:56.697855 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1178c0a-2720-4600-805e-893273c2c1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:56.698039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1178c0a-2720-4600-805e-893273c2c1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:56.772075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-d1178c0a-2720-4600-805e-893273c2c1c9 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:56.773274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d1178c0a-2720-4600-805e-893273c2c1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:56.814788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:56.869519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 542/1111] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:57.022398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41987e2-cfc5-43c6-b0e4-e27470b4738b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:57.023853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b41987e2-cfc5-43c6-b0e4-e27470b4738b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:57.169070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41987e2-cfc5-43c6-b0e4-e27470b4738b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:57.222581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:57.281816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a414e06-baf0-46bf-bb11-6baee9df43bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:57.282355 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a414e06-baf0-46bf-bb11-6baee9df43bb service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:57.282628 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a414e06-baf0-46bf-bb11-6baee9df43bb service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:57.282895 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a414e06-baf0-46bf-bb11-6baee9df43bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:57.284508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6a414e06-baf0-46bf-bb11-6baee9df43bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:57.373413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a414e06-baf0-46bf-bb11-6baee9df43bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:57.374994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a414e06-baf0-46bf-bb11-6baee9df43bb service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:57.393129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 570/1112] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:48:58.448280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a2673-58f8-4bdc-937b-76d850969eca None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:58.448853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a2673-58f8-4bdc-937b-76d850969eca None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:58.448853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a2673-58f8-4bdc-937b-76d850969eca None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:58.450121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 543/1113] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:48:58.456439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a56efb-57e6-489c-ad94-bf61a654d681 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:58.456902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a56efb-57e6-489c-ad94-bf61a654d681 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:58.456999 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a56efb-57e6-489c-ad94-bf61a654d681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:58.532392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-b8a56efb-57e6-489c-ad94-bf61a654d681 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:48:58.533578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b8a56efb-57e6-489c-ad94-bf61a654d681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:58.535550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:58.631386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 571/1114] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:48:58.781518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a2673-58f8-4bdc-937b-76d850969eca None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:48:58.783073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a32a2673-58f8-4bdc-937b-76d850969eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:58.931250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a2673-58f8-4bdc-937b-76d850969eca None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:48:58.957646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:48:59.047510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a15d74-6fd3-4a67-9654-ba30bc564732 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:48:59.048152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a15d74-6fd3-4a67-9654-ba30bc564732 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:48:59.048428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a15d74-6fd3-4a67-9654-ba30bc564732 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:48:59.048695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a15d74-6fd3-4a67-9654-ba30bc564732 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:48:59.050290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f1a15d74-6fd3-4a67-9654-ba30bc564732 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:48:59.139837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a15d74-6fd3-4a67-9654-ba30bc564732 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:48:59.141452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a15d74-6fd3-4a67-9654-ba30bc564732 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:48:59.159037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 544/1115] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:01.215907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2335-24dd-4e72-b049-48e77b4c568f None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:01.216490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2335-24dd-4e72-b049-48e77b4c568f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:01.216490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2335-24dd-4e72-b049-48e77b4c568f None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:01.217777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 572/1116] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:01.223193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0da17-171e-42f2-9e2e-05410ee53c3e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:01.223757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0da17-171e-42f2-9e2e-05410ee53c3e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:01.223757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0da17-171e-42f2-9e2e-05410ee53c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:01.264009 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:01.290130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6ff0da17-171e-42f2-9e2e-05410ee53c3e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:01.291335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6ff0da17-171e-42f2-9e2e-05410ee53c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:01.387803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 545/1117] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:49:01.673873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e2335-24dd-4e72-b049-48e77b4c568f None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:01.675420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2c3e2335-24dd-4e72-b049-48e77b4c568f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:01.775746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e2335-24dd-4e72-b049-48e77b4c568f None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:01.807711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:01.848576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0a102-29f2-496e-8f3b-92a2ac9f0108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:01.849259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0a102-29f2-496e-8f3b-92a2ac9f0108 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:01.849530 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0a102-29f2-496e-8f3b-92a2ac9f0108 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:01.849795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0a102-29f2-496e-8f3b-92a2ac9f0108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:01.851363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-63c0a102-29f2-496e-8f3b-92a2ac9f0108 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:01.959613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0a102-29f2-496e-8f3b-92a2ac9f0108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:01.961009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0a102-29f2-496e-8f3b-92a2ac9f0108 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:01.979807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 573/1118] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:04.334023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476ba15-ccc6-4304-b7d0-04408afa23c4 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:04.334569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476ba15-ccc6-4304-b7d0-04408afa23c4 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:04.334569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476ba15-ccc6-4304-b7d0-04408afa23c4 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:04.335859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 546/1119] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:49:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:04.341039 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcbd60-df35-4e18-abfd-e82cc55ec2ce None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:04.341495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcbd60-df35-4e18-abfd-e82cc55ec2ce None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:04.341598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcbd60-df35-4e18-abfd-e82cc55ec2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:04.409719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7bfcbd60-df35-4e18-abfd-e82cc55ec2ce None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:04.411581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7bfcbd60-df35-4e18-abfd-e82cc55ec2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:04.424343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:04.507710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 574/1120] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:04.789496 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476ba15-ccc6-4304-b7d0-04408afa23c4 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:04.791043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c476ba15-ccc6-4304-b7d0-04408afa23c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:04.892831 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476ba15-ccc6-4304-b7d0-04408afa23c4 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:04.906438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:04.968752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0be5b-4c0b-4fac-89b9-ea234462f901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:04.969381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0be5b-4c0b-4fac-89b9-ea234462f901 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:04.969654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0be5b-4c0b-4fac-89b9-ea234462f901 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:04.969927 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0be5b-4c0b-4fac-89b9-ea234462f901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:04.971504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c5f0be5b-4c0b-4fac-89b9-ea234462f901 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:05.081507 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0be5b-4c0b-4fac-89b9-ea234462f901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:05.083691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0be5b-4c0b-4fac-89b9-ea234462f901 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:05.101126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 547/1121] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:07.554615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c7049-d0cc-47b4-a6d6-8ba4a247c7c7 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:07.555168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c7049-d0cc-47b4-a6d6-8ba4a247c7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:07.555261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c7049-d0cc-47b4-a6d6-8ba4a247c7c7 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:07.556591 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 575/1122] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:07.561765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7af5d-4157-440e-837a-bebd949b76f3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:07.562144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7af5d-4157-440e-837a-bebd949b76f3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:07.562311 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7af5d-4157-440e-837a-bebd949b76f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:07.610027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:07.628444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-51e7af5d-4157-440e-837a-bebd949b76f3 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:07.630213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-51e7af5d-4157-440e-837a-bebd949b76f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:07.727801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 548/1123] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:08.012498 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c7049-d0cc-47b4-a6d6-8ba4a247c7c7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:08.014067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b14c7049-d0cc-47b4-a6d6-8ba4a247c7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:08.116062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c7049-d0cc-47b4-a6d6-8ba4a247c7c7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:08.155805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:08.189766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8edb6-7941-4c04-b289-f5a566947245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:08.190364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8edb6-7941-4c04-b289-f5a566947245 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:08.190638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8edb6-7941-4c04-b289-f5a566947245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:08.190905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8edb6-7941-4c04-b289-f5a566947245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:08.193044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2fa8edb6-7941-4c04-b289-f5a566947245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:08.299488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8edb6-7941-4c04-b289-f5a566947245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:08.301545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8edb6-7941-4c04-b289-f5a566947245 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:08.320700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 576/1124] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:10.596752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2285-2b21-435e-9cd3-caa46f8e04c6 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:10.597301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2285-2b21-435e-9cd3-caa46f8e04c6 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:10.597301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2285-2b21-435e-9cd3-caa46f8e04c6 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:10.598598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 549/1125] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:10.603964 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0849c-c036-44fb-af74-1e820aef1b0e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:10.604384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0849c-c036-44fb-af74-1e820aef1b0e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:10.604512 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0849c-c036-44fb-af74-1e820aef1b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:10.670288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-09d0849c-c036-44fb-af74-1e820aef1b0e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:10.671486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-09d0849c-c036-44fb-af74-1e820aef1b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:10.693208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:10.769913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 577/1126] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:11.059859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac2285-2b21-435e-9cd3-caa46f8e04c6 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:11.061493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-77ac2285-2b21-435e-9cd3-caa46f8e04c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:11.167012 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac2285-2b21-435e-9cd3-caa46f8e04c6 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:11.186942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:11.240699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cfa9a-9a04-455c-bde8-7add88d6d68b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:11.241296 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cfa9a-9a04-455c-bde8-7add88d6d68b service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:11.241570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cfa9a-9a04-455c-bde8-7add88d6d68b service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:11.241834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cfa9a-9a04-455c-bde8-7add88d6d68b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:11.243416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5f9cfa9a-9a04-455c-bde8-7add88d6d68b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:11.352832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cfa9a-9a04-455c-bde8-7add88d6d68b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:11.354268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cfa9a-9a04-455c-bde8-7add88d6d68b service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:11.372259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 550/1127] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:13.665937 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f38df4-5a63-477e-8857-11ef1e3d5ac0 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:13.666537 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f38df4-5a63-477e-8857-11ef1e3d5ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:13.666537 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f38df4-5a63-477e-8857-11ef1e3d5ac0 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:13.667811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 578/1128] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:13.673200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62618e7b-c86e-4c59-a26f-8feb5ddc4d4b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:13.673608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62618e7b-c86e-4c59-a26f-8feb5ddc4d4b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:13.673740 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62618e7b-c86e-4c59-a26f-8feb5ddc4d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:13.736973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:13.740704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-62618e7b-c86e-4c59-a26f-8feb5ddc4d4b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:13.742243 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-62618e7b-c86e-4c59-a26f-8feb5ddc4d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:13.876288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 551/1129] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:14.225273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f38df4-5a63-477e-8857-11ef1e3d5ac0 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:14.226787 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-59f38df4-5a63-477e-8857-11ef1e3d5ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:14.331182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f38df4-5a63-477e-8857-11ef1e3d5ac0 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:14.388097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:14.426024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fe83c-d466-41d1-a97c-19842fdd06f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:14.426024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fe83c-d466-41d1-a97c-19842fdd06f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:14.426024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fe83c-d466-41d1-a97c-19842fdd06f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:14.426024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fe83c-d466-41d1-a97c-19842fdd06f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:14.428469 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ac7fe83c-d466-41d1-a97c-19842fdd06f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:14.538700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fe83c-d466-41d1-a97c-19842fdd06f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:14.540060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fe83c-d466-41d1-a97c-19842fdd06f7 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:14.557347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 579/1130] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:16.748445 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9cea-3736-487f-8324-0e54184c587a None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:16.748969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9cea-3736-487f-8324-0e54184c587a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:16.748969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9cea-3736-487f-8324-0e54184c587a None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:16.750210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 552/1131] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:16.756731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb0943-7130-433e-b50c-fbfaeb4eae32 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:16.757159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb0943-7130-433e-b50c-fbfaeb4eae32 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:16.757265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb0943-7130-433e-b50c-fbfaeb4eae32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:16.822646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-deeb0943-7130-433e-b50c-fbfaeb4eae32 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:16.823836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-deeb0943-7130-433e-b50c-fbfaeb4eae32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:16.852601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:16.921238 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 580/1132] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:17.205188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b9cea-3736-487f-8324-0e54184c587a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:17.206748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-517b9cea-3736-487f-8324-0e54184c587a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:17.309188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b9cea-3736-487f-8324-0e54184c587a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:17.338143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:17.384392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676a57b-1036-402a-9ac9-666225cc15e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:17.384986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676a57b-1036-402a-9ac9-666225cc15e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:17.385257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676a57b-1036-402a-9ac9-666225cc15e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:17.385547 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676a57b-1036-402a-9ac9-666225cc15e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:17.387129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a676a57b-1036-402a-9ac9-666225cc15e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:17.498830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676a57b-1036-402a-9ac9-666225cc15e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:17.500191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676a57b-1036-402a-9ac9-666225cc15e2 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:17.517801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 553/1133] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:19.777441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf0cad-7e82-498c-9e63-793a97d69b56 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:19.777952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf0cad-7e82-498c-9e63-793a97d69b56 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:19.777952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf0cad-7e82-498c-9e63-793a97d69b56 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:19.779208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 581/1134] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:19.785221 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847271e-c94b-4cda-8cf9-438fcd27f032 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:19.785670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847271e-c94b-4cda-8cf9-438fcd27f032 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:19.785774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847271e-c94b-4cda-8cf9-438fcd27f032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:19.851352 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2847271e-c94b-4cda-8cf9-438fcd27f032 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:19.852580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2847271e-c94b-4cda-8cf9-438fcd27f032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:19.860833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:19.949937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 554/1135] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:20.236443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf0cad-7e82-498c-9e63-793a97d69b56 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:20.238748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6eaf0cad-7e82-498c-9e63-793a97d69b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:20.341181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf0cad-7e82-498c-9e63-793a97d69b56 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:20.397777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:20.417394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d421840-0643-41f0-bdc9-f5b570b3b4ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:20.417986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d421840-0643-41f0-bdc9-f5b570b3b4ee service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:20.418252 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d421840-0643-41f0-bdc9-f5b570b3b4ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:20.418514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d421840-0643-41f0-bdc9-f5b570b3b4ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:20.420154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9d421840-0643-41f0-bdc9-f5b570b3b4ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:20.531554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d421840-0643-41f0-bdc9-f5b570b3b4ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:20.533026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d421840-0643-41f0-bdc9-f5b570b3b4ee service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:20.554005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 582/1136] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:22.741518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2cde4-5a4e-409f-9f03-154e6c33b1c7 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:22.742053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2cde4-5a4e-409f-9f03-154e6c33b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:22.742053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2cde4-5a4e-409f-9f03-154e6c33b1c7 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:22.743408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 555/1137] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:22.748759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d3c7-3335-40bf-94c7-0e2eec80c7c0 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:22.749143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d3c7-3335-40bf-94c7-0e2eec80c7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:22.749297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d3c7-3335-40bf-94c7-0e2eec80c7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:22.815077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8927d3c7-3335-40bf-94c7-0e2eec80c7c0 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:22.816441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8927d3c7-3335-40bf-94c7-0e2eec80c7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:22.854311 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:22.912427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 583/1138] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:23.196149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2cde4-5a4e-409f-9f03-154e6c33b1c7 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:23.197696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5af2cde4-5a4e-409f-9f03-154e6c33b1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:23.302474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2cde4-5a4e-409f-9f03-154e6c33b1c7 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:23.340347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:23.376826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a4928-6265-451a-b09e-431b627061d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:23.377609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4928-6265-451a-b09e-431b627061d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:23.378312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4928-6265-451a-b09e-431b627061d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:23.378584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4928-6265-451a-b09e-431b627061d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:23.380259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2e5a4928-6265-451a-b09e-431b627061d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:23.490325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a4928-6265-451a-b09e-431b627061d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:23.491690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a4928-6265-451a-b09e-431b627061d8 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:23.510054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 556/1139] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:25.683723 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cbd08-17e4-45db-ae66-bc52ad62b7d6 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:25.684403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cbd08-17e4-45db-ae66-bc52ad62b7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:25.684403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cbd08-17e4-45db-ae66-bc52ad62b7d6 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:25.685534 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 584/1140] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:25.691826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afba479-8cae-4c29-a79f-eb22c5b69e96 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:25.692300 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afba479-8cae-4c29-a79f-eb22c5b69e96 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:25.692424 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afba479-8cae-4c29-a79f-eb22c5b69e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:25.757943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6afba479-8cae-4c29-a79f-eb22c5b69e96 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:25.759143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6afba479-8cae-4c29-a79f-eb22c5b69e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:25.777616 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:25.856727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 557/1141] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:49:26.156778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32865e60-d60a-49dd-8fed-0abe477dd0a5 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:26.157058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32865e60-d60a-49dd-8fed-0abe477dd0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:26.157357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32865e60-d60a-49dd-8fed-0abe477dd0a5 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:26.158728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 585/1142] 10.222.0.27 () {54 vars in 964 bytes} [Fri Oct 14 17:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:26.164961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a7ce3-a2cf-44e3-abe9-4bba241df8eb None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:26.165246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a7ce3-a2cf-44e3-abe9-4bba241df8eb None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:26.165513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a7ce3-a2cf-44e3-abe9-4bba241df8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:26.240387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-652a7ce3-a2cf-44e3-abe9-4bba241df8eb None None] MFA Rules not processed for user `3c17f274e4c14f2f99650844642437a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:26.241502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-652a7ce3-a2cf-44e3-abe9-4bba241df8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:26.316711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 558/1143] 10.222.0.27 () {58 vars in 1061 bytes} [Fri Oct 14 17:49:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:26.325642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32865e60-d60a-49dd-8fed-0abe477dd0a5 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:26.327184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-32865e60-d60a-49dd-8fed-0abe477dd0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:26.431814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32865e60-d60a-49dd-8fed-0abe477dd0a5 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:26.499944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:26.507140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea252c2-60ca-4116-b9c1-7088e6245477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:26.507732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea252c2-60ca-4116-b9c1-7088e6245477 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:26.508033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea252c2-60ca-4116-b9c1-7088e6245477 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:26.508299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea252c2-60ca-4116-b9c1-7088e6245477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:26.509868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1ea252c2-60ca-4116-b9c1-7088e6245477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:26.617281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea252c2-60ca-4116-b9c1-7088e6245477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:26.618717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea252c2-60ca-4116-b9c1-7088e6245477 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:26.636350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 586/1144] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:29.206090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106db6a-a98c-4c7b-a552-50c7b910619c None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:29.206652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106db6a-a98c-4c7b-a552-50c7b910619c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:29.206652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106db6a-a98c-4c7b-a552-50c7b910619c None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:29.207957 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 559/1145] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:29.213346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddced56-7a64-434a-a1c6-ad27367deb3d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:29.213795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddced56-7a64-434a-a1c6-ad27367deb3d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:29.213923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddced56-7a64-434a-a1c6-ad27367deb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:29.280445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-3ddced56-7a64-434a-a1c6-ad27367deb3d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:29.281561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3ddced56-7a64-434a-a1c6-ad27367deb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:29.329082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:29.377438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 587/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:29.657764 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106db6a-a98c-4c7b-a552-50c7b910619c None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:29.659621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3106db6a-a98c-4c7b-a552-50c7b910619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:29.674638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:29.765538 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106db6a-a98c-4c7b-a552-50c7b910619c None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:29.839767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd20e9-a0e4-4510-bbef-00c99f853245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:29.840392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd20e9-a0e4-4510-bbef-00c99f853245 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:29.840667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd20e9-a0e4-4510-bbef-00c99f853245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:29.840939 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd20e9-a0e4-4510-bbef-00c99f853245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:29.842527 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e4fd20e9-a0e4-4510-bbef-00c99f853245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:29.910045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:29.951104 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd20e9-a0e4-4510-bbef-00c99f853245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:29.952552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd20e9-a0e4-4510-bbef-00c99f853245 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:29.970146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 560/1147] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:39.375819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=22 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:39.967911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=25 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:50.163627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667745ae-ca8d-477f-ae3b-55d802a08f9b None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:50.164233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667745ae-ca8d-477f-ae3b-55d802a08f9b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:50.164233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667745ae-ca8d-477f-ae3b-55d802a08f9b None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:50.165523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 588/1148] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:50.170760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd619621-e94e-45a1-b78e-b402e7d0f55d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:50.171154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd619621-e94e-45a1-b78e-b402e7d0f55d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:50.171292 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd619621-e94e-45a1-b78e-b402e7d0f55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:50.237937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-bd619621-e94e-45a1-b78e-b402e7d0f55d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:50.239113 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-bd619621-e94e-45a1-b78e-b402e7d0f55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:50.337183 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 561/1149] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:50.658046 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b563b0e-a7ac-4154-9c9f-ed4597a90b3c None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:50.658322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b563b0e-a7ac-4154-9c9f-ed4597a90b3c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:50.658591 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b563b0e-a7ac-4154-9c9f-ed4597a90b3c None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:50.659952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 589/1150] 10.222.0.27 () {54 vars in 962 bytes} [Fri Oct 14 17:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:49:50.664954 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740976d-994b-4a16-a1eb-226835218f51 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:50.665230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740976d-994b-4a16-a1eb-226835218f51 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:50.665496 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740976d-994b-4a16-a1eb-226835218f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:50.741994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1740976d-994b-4a16-a1eb-226835218f51 None None] MFA Rules not processed for user `ae3566d85c9c4c559ba672269ab38f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:50.743101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1740976d-994b-4a16-a1eb-226835218f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:50.763479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:50.819217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 562/1151] 10.222.0.27 () {58 vars in 1059 bytes} [Fri Oct 14 17:49:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:50.826868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b563b0e-a7ac-4154-9c9f-ed4597a90b3c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:50.828528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7b563b0e-a7ac-4154-9c9f-ed4597a90b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:50.930126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b563b0e-a7ac-4154-9c9f-ed4597a90b3c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:51.003472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabfd68-2494-4ffe-bb17-f5e51bfc9c1d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3566d85c9c4c559ba672269ab38f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:51.004012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabfd68-2494-4ffe-bb17-f5e51bfc9c1d service glance] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:51.004292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabfd68-2494-4ffe-bb17-f5e51bfc9c1d service glance] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:51.004555 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabfd68-2494-4ffe-bb17-f5e51bfc9c1d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:51.006137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7dabfd68-2494-4ffe-bb17-f5e51bfc9c1d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:51.058677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:51.114618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabfd68-2494-4ffe-bb17-f5e51bfc9c1d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:51.116098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabfd68-2494-4ffe-bb17-f5e51bfc9c1d service glance] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:51.134271 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 590/1152] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:53.366589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31c499-f446-461c-a3ab-d3bb5db686de None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:53.367056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31c499-f446-461c-a3ab-d3bb5db686de None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:53.367171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31c499-f446-461c-a3ab-d3bb5db686de None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:53.368607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 563/1153] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:53.373840 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4da336-e388-4ff5-a574-255db005821a None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:53.374291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4da336-e388-4ff5-a574-255db005821a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:53.374396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4da336-e388-4ff5-a574-255db005821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:53.441014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-eb4da336-e388-4ff5-a574-255db005821a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:53.442238 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-eb4da336-e388-4ff5-a574-255db005821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:53.537118 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 591/1154] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:53.808148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31c499-f446-461c-a3ab-d3bb5db686de None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:53.809681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6d31c499-f446-461c-a3ab-d3bb5db686de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:53.934471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31c499-f446-461c-a3ab-d3bb5db686de None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:53.983777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:54.029714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8a6ec-5fe1-44c3-810f-7cc0cb0b84a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:54.030306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8a6ec-5fe1-44c3-810f-7cc0cb0b84a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:54.030577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8a6ec-5fe1-44c3-810f-7cc0cb0b84a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:54.030840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8a6ec-5fe1-44c3-810f-7cc0cb0b84a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:54.031969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8a6ec-5fe1-44c3-810f-7cc0cb0b84a8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:54.033238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: WARNING py.warnings [None req-a4d8a6ec-5fe1-44c3-810f-7cc0cb0b84a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:49:54.033238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: warnings.warn(msg) Oct 14 17:49:54.033238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: Oct 14 17:49:54.034366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8a6ec-5fe1-44c3-810f-7cc0cb0b84a8 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:54.053774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 564/1155] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Oct 14 17:49:53 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:49:56.219169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4a05-2604-48fc-8b58-51f465ecfefe None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:56.219778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4a05-2604-48fc-8b58-51f465ecfefe None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:56.219778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4a05-2604-48fc-8b58-51f465ecfefe None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:56.221092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 592/1156] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:56.226845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22529711-98e9-40e7-82bd-8d6739e239d4 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:56.227666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22529711-98e9-40e7-82bd-8d6739e239d4 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:56.227963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22529711-98e9-40e7-82bd-8d6739e239d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:56.292686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-22529711-98e9-40e7-82bd-8d6739e239d4 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:56.293870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-22529711-98e9-40e7-82bd-8d6739e239d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:56.392512 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 565/1157] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:56.684522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4a05-2604-48fc-8b58-51f465ecfefe None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:56.686091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-72ea4a05-2604-48fc-8b58-51f465ecfefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:56.696848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:56.834501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4a05-2604-48fc-8b58-51f465ecfefe None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:49:56.913404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:56.946898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c30fe-cf83-43fb-9d88-dda9f34b167e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:49:56.947433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c30fe-cf83-43fb-9d88-dda9f34b167e service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:56.947706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c30fe-cf83-43fb-9d88-dda9f34b167e service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:56.948071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c30fe-cf83-43fb-9d88-dda9f34b167e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:56.949664 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-591c30fe-cf83-43fb-9d88-dda9f34b167e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:57.039087 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c30fe-cf83-43fb-9d88-dda9f34b167e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:49:57.040719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c30fe-cf83-43fb-9d88-dda9f34b167e service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:49:57.060576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 593/1158] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:49:59.391684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0924a-7f33-4e50-893b-9ddf500ef65b None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:59.392244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0924a-7f33-4e50-893b-9ddf500ef65b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:59.392244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0924a-7f33-4e50-893b-9ddf500ef65b None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:59.393581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 566/1159] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:49:59.398990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13b2fa-8617-4a37-9dd1-15bb3b28d366 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:49:59.399404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13b2fa-8617-4a37-9dd1-15bb3b28d366 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:49:59.399556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13b2fa-8617-4a37-9dd1-15bb3b28d366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:49:59.469872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4e13b2fa-8617-4a37-9dd1-15bb3b28d366 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:49:59.471057 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4e13b2fa-8617-4a37-9dd1-15bb3b28d366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:59.519377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:59.570917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 594/1160] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:49:59.862656 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0924a-7f33-4e50-893b-9ddf500ef65b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:49:59.864242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f9c0924a-7f33-4e50-893b-9ddf500ef65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:49:59.889845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:49:59.970194 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0924a-7f33-4e50-893b-9ddf500ef65b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:50:00.041374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0d0c3-8ba6-42e8-b895-934a5ba56ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:50:00.041973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0d0c3-8ba6-42e8-b895-934a5ba56ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:50:00.042245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0d0c3-8ba6-42e8-b895-934a5ba56ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:50:00.042512 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0d0c3-8ba6-42e8-b895-934a5ba56ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:50:00.044129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3dc0d0c3-8ba6-42e8-b895-934a5ba56ce1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:50:00.118265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:50:00.153299 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0d0c3-8ba6-42e8-b895-934a5ba56ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:50:00.154720 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0d0c3-8ba6-42e8-b895-934a5ba56ce1 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:50:00.173735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 567/1161] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:50:02.383732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71253550-d90c-4713-9e12-f48c507685e7 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:50:02.384291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71253550-d90c-4713-9e12-f48c507685e7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:50:02.384291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71253550-d90c-4713-9e12-f48c507685e7 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:50:02.385632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 595/1162] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:50:02.391303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdeb8f-4f37-4cea-be55-045658e98096 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:50:02.391760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdeb8f-4f37-4cea-be55-045658e98096 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:50:02.391867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdeb8f-4f37-4cea-be55-045658e98096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:50:02.458280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-cbfdeb8f-4f37-4cea-be55-045658e98096 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:50:02.459465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-cbfdeb8f-4f37-4cea-be55-045658e98096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:50:02.555425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 568/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:50:02.853180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71253550-d90c-4713-9e12-f48c507685e7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:50:02.854737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-71253550-d90c-4713-9e12-f48c507685e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:50:02.958694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71253550-d90c-4713-9e12-f48c507685e7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:50:03.031572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:50:03.033669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59963b-de7c-4ac5-9a6b-a763a7df1eab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:50:03.034309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59963b-de7c-4ac5-9a6b-a763a7df1eab service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:50:03.034614 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59963b-de7c-4ac5-9a6b-a763a7df1eab service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:50:03.034924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59963b-de7c-4ac5-9a6b-a763a7df1eab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:50:03.037072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3e59963b-de7c-4ac5-9a6b-a763a7df1eab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:50:03.148066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59963b-de7c-4ac5-9a6b-a763a7df1eab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:50:03.149432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59963b-de7c-4ac5-9a6b-a763a7df1eab service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:50:03.167672 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 596/1164] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:50:05.492708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0a7db-ecc8-48c2-ae73-02921bcfe5ba None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:50:05.493278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0a7db-ecc8-48c2-ae73-02921bcfe5ba None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:50:05.493278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0a7db-ecc8-48c2-ae73-02921bcfe5ba None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:50:05.494578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 569/1165] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:50:05.501150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f953cc8-9fe1-4a33-a7f3-eb99f56b6de6 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:50:05.501576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f953cc8-9fe1-4a33-a7f3-eb99f56b6de6 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:50:05.501721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f953cc8-9fe1-4a33-a7f3-eb99f56b6de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:50:05.567333 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-3f953cc8-9fe1-4a33-a7f3-eb99f56b6de6 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:50:05.568564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3f953cc8-9fe1-4a33-a7f3-eb99f56b6de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:50:05.619691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:50:05.663799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 597/1166] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:50:05.955757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0a7db-ecc8-48c2-ae73-02921bcfe5ba None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:50:05.957310 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d1a0a7db-ecc8-48c2-ae73-02921bcfe5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:50:05.993609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:50:06.064048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0a7db-ecc8-48c2-ae73-02921bcfe5ba None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:50:06.138553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a738d7-f46e-4f4a-8a3a-47394d9c392f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:50:06.139204 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a738d7-f46e-4f4a-8a3a-47394d9c392f service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:50:06.139504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a738d7-f46e-4f4a-8a3a-47394d9c392f service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:50:06.139773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a738d7-f46e-4f4a-8a3a-47394d9c392f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:50:06.141385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-19a738d7-f46e-4f4a-8a3a-47394d9c392f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:50:06.221104 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:50:06.250269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a738d7-f46e-4f4a-8a3a-47394d9c392f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:50:06.251694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a738d7-f46e-4f4a-8a3a-47394d9c392f service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:50:06.269431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 570/1167] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:50:08.652847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab001ee-5111-4832-a9ab-d3716c6b4893 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:50:08.653379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab001ee-5111-4832-a9ab-d3716c6b4893 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:50:08.653379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab001ee-5111-4832-a9ab-d3716c6b4893 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:50:08.654633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 598/1168] 10.222.0.27 () {54 vars in 927 bytes} [Fri Oct 14 17:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:50:08.660425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371db86b-717e-4160-8892-1acf153a1682 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:50:08.660839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371db86b-717e-4160-8892-1acf153a1682 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:50:08.660949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371db86b-717e-4160-8892-1acf153a1682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:50:08.733090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-371db86b-717e-4160-8892-1acf153a1682 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:50:08.734294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-371db86b-717e-4160-8892-1acf153a1682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:50:08.830063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 571/1169] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Oct 14 17:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:50:09.125056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab001ee-5111-4832-a9ab-d3716c6b4893 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:50:09.126615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-aab001ee-5111-4832-a9ab-d3716c6b4893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:50:09.231110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab001ee-5111-4832-a9ab-d3716c6b4893 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:50:09.305911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73362b6f-a78a-4b73-a762-41deaa2e439a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:50:09.306506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73362b6f-a78a-4b73-a762-41deaa2e439a service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:50:09.306776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73362b6f-a78a-4b73-a762-41deaa2e439a service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:50:09.307039 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73362b6f-a78a-4b73-a762-41deaa2e439a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:50:09.308635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-73362b6f-a78a-4b73-a762-41deaa2e439a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:50:09.317456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:50:09.417367 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73362b6f-a78a-4b73-a762-41deaa2e439a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:50:09.418797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73362b6f-a78a-4b73-a762-41deaa2e439a service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:50:09.437213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 599/1170] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:50:18.828490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=82 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:50:19.434924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=50 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:29.094455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115dcf3-36f3-4027-af31-82b4b2009e5b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:29.095047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115dcf3-36f3-4027-af31-82b4b2009e5b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:29.095047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115dcf3-36f3-4027-af31-82b4b2009e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:29.180592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-3115dcf3-36f3-4027-af31-82b4b2009e5b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:51:29.181841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3115dcf3-36f3-4027-af31-82b4b2009e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:29.275839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 572/1171] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:51:29.284227 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6739aad-aac9-4cc0-987d-6a631a81a4f6 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:29.284703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6739aad-aac9-4cc0-987d-6a631a81a4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:29.284820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6739aad-aac9-4cc0-987d-6a631a81a4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:29.361115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-a6739aad-aac9-4cc0-987d-6a631a81a4f6 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:51:29.362349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a6739aad-aac9-4cc0-987d-6a631a81a4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:29.457467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 600/1172] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:51:29.462154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115dcf3-36f3-4027-af31-82b4b2009e5b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:51:29.464151 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3115dcf3-36f3-4027-af31-82b4b2009e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:29.475360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115dcf3-36f3-4027-af31-82b4b2009e5b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:51:29.537185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:29.570218 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:51:29.570815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:29.571083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:29.571345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:29.571998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:51:29.572804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:51:29.573857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: WARNING py.warnings [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:51:29.573857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: warnings.warn(msg) Oct 14 17:51:29.573857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: Oct 14 17:51:29.574731 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:51:29.585545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 573/1173] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:51:29.596064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6739aad-aac9-4cc0-987d-6a631a81a4f6 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:51:29.605922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6739aad-aac9-4cc0-987d-6a631a81a4f6 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:51:29.667005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:29.700375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340af5a0-0115-4f5d-98e2-bfd5b54423a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:51:29.701008 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340af5a0-0115-4f5d-98e2-bfd5b54423a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:29.701284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340af5a0-0115-4f5d-98e2-bfd5b54423a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:29.701554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340af5a0-0115-4f5d-98e2-bfd5b54423a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:29.702174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340af5a0-0115-4f5d-98e2-bfd5b54423a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:51:29.703283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WARNING py.warnings [None req-340af5a0-0115-4f5d-98e2-bfd5b54423a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:51:29.703283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: warnings.warn(msg) Oct 14 17:51:29.703283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: Oct 14 17:51:29.704504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340af5a0-0115-4f5d-98e2-bfd5b54423a5 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:51:29.760644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 601/1174] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:51:29 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:51:29.764900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:51:29.766425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:29.799781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac856-0ab3-4f34-a16c-55565a992ccf admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:51:29.892365 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b802fd7-9063-49f9-a325-7997e58177f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:51:29.892964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b802fd7-9063-49f9-a325-7997e58177f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:29.893238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b802fd7-9063-49f9-a325-7997e58177f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:29.893506 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b802fd7-9063-49f9-a325-7997e58177f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:29.894188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b802fd7-9063-49f9-a325-7997e58177f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:51:29.895194 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: WARNING py.warnings [None req-2b802fd7-9063-49f9-a325-7997e58177f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:51:29.895194 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: warnings.warn(msg) Oct 14 17:51:29.895194 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: Oct 14 17:51:29.896435 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b802fd7-9063-49f9-a325-7997e58177f6 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:51:29.939534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 574/1175] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:51:29 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:51:29.943587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340af5a0-0115-4f5d-98e2-bfd5b54423a5 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:51:29.945144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-340af5a0-0115-4f5d-98e2-bfd5b54423a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:30.006792 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340af5a0-0115-4f5d-98e2-bfd5b54423a5 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:51:30.088817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:30.099185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca82dca-5df4-4517-acfb-67d0aa6d9400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:51:30.099781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca82dca-5df4-4517-acfb-67d0aa6d9400 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:30.100082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca82dca-5df4-4517-acfb-67d0aa6d9400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:30.100351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca82dca-5df4-4517-acfb-67d0aa6d9400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:30.100979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca82dca-5df4-4517-acfb-67d0aa6d9400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:51:30.102579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca82dca-5df4-4517-acfb-67d0aa6d9400 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:51:30.114735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 602/1176] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:51:30.118775 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b802fd7-9063-49f9-a325-7997e58177f6 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:51:30.120923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2b802fd7-9063-49f9-a325-7997e58177f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:30.132232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b802fd7-9063-49f9-a325-7997e58177f6 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:51:30.157873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:30.227503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c36fd91-8000-412f-8f6e-a4246ab9dae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:51:30.228193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36fd91-8000-412f-8f6e-a4246ab9dae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:30.229067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36fd91-8000-412f-8f6e-a4246ab9dae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:30.229443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36fd91-8000-412f-8f6e-a4246ab9dae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:30.230079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c36fd91-8000-412f-8f6e-a4246ab9dae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:51:30.231681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c36fd91-8000-412f-8f6e-a4246ab9dae1 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:51:30.243700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 575/1177] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:51:30.248411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca82dca-5df4-4517-acfb-67d0aa6d9400 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:51:30.249925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2ca82dca-5df4-4517-acfb-67d0aa6d9400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:30.282739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca82dca-5df4-4517-acfb-67d0aa6d9400 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:51:30.374960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3677a7-1c51-4142-983f-9b7a8b29bb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:51:30.375613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3677a7-1c51-4142-983f-9b7a8b29bb18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:30.375907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3677a7-1c51-4142-983f-9b7a8b29bb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:30.376183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3677a7-1c51-4142-983f-9b7a8b29bb18 admin admin] PATH_INFO: `/v3/projects/c8b10aa4ce6d475395f22df81f9b0da0/users/5f0ae290cb1e4382bb164dd260acd759/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:30.398933 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3677a7-1c51-4142-983f-9b7a8b29bb18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8b10aa4ce6d475395f22df81f9b0da0, user_id=5f0ae290cb1e4382bb164dd260acd759, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:51:30.402210 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3677a7-1c51-4142-983f-9b7a8b29bb18 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:51:30.426956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 603/1178] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:51:30 2022] PUT /identity/v3/projects/c8b10aa4ce6d475395f22df81f9b0da0/users/5f0ae290cb1e4382bb164dd260acd759/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:51:30.438801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa108-31d3-492d-bef4-8d234c7379f3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:30.439085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa108-31d3-492d-bef4-8d234c7379f3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:30.439382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa108-31d3-492d-bef4-8d234c7379f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:30.497801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-0f7fa108-31d3-492d-bef4-8d234c7379f3 None None] MFA Rules not processed for user `5f0ae290cb1e4382bb164dd260acd759`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:51:30.499056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0f7fa108-31d3-492d-bef4-8d234c7379f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:30.576318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 576/1179] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:51:30.590671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d85cc-d1d5-42ba-b714-41045cd7d400 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:30.590954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d85cc-d1d5-42ba-b714-41045cd7d400 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:30.591228 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d85cc-d1d5-42ba-b714-41045cd7d400 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:30.592586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 604/1180] 10.222.0.27 () {54 vars in 963 bytes} [Fri Oct 14 17:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:51:30.597422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649d90-366c-4524-9b28-d9bad0045382 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:30.597704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649d90-366c-4524-9b28-d9bad0045382 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:30.597976 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649d90-366c-4524-9b28-d9bad0045382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:30.630013 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:30.674771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1e649d90-366c-4524-9b28-d9bad0045382 None None] MFA Rules not processed for user `8f51ee9e45fb4dda8d5b9740aa0ac828`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:51:30.675993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1e649d90-366c-4524-9b28-d9bad0045382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:30.751465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 577/1181] 10.222.0.27 () {58 vars in 1060 bytes} [Fri Oct 14 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:51:30.759079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d85cc-d1d5-42ba-b714-41045cd7d400 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:51:30.760682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-bb0d85cc-d1d5-42ba-b714-41045cd7d400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:30.810438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:30.864609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d85cc-d1d5-42ba-b714-41045cd7d400 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:51:30.937900 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d978348-388a-4244-bfca-17d02a0e412b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:51:30.938418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d978348-388a-4244-bfca-17d02a0e412b service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:30.938690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d978348-388a-4244-bfca-17d02a0e412b service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:30.938952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d978348-388a-4244-bfca-17d02a0e412b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:30.940538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6d978348-388a-4244-bfca-17d02a0e412b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:31.037420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d978348-388a-4244-bfca-17d02a0e412b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:51:31.038852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d978348-388a-4244-bfca-17d02a0e412b service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:51:31.042271 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:31.057441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 605/1182] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:51:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:51:31.104363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e54814-cb66-464d-8a33-baee9579570a None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:31.104648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e54814-cb66-464d-8a33-baee9579570a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:31.104922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e54814-cb66-464d-8a33-baee9579570a None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:31.106189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 578/1183] 10.222.0.27 () {52 vars in 830 bytes} [Fri Oct 14 17:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:51:31.109860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d978348-388a-4244-bfca-17d02a0e412b service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:51:31.111072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6d978348-388a-4244-bfca-17d02a0e412b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:31.210224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d978348-388a-4244-bfca-17d02a0e412b service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:51:31.302321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:51:31.302928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:31.303198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:31.303461 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] PATH_INFO: `/v3/users/5f0ae290cb1e4382bb164dd260acd759` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:31.316529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0ae290cb1e4382bb164dd260acd759)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:51:31.317544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WARNING py.warnings [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:51:31.317544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: warnings.warn(msg) Oct 14 17:51:31.317544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: Oct 14 17:51:31.318735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:51:31.329033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:31.389418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0ae290cb1e4382bb164dd260acd759'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:51:31.390059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0ae290cb1e4382bb164dd260acd759'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:51:31.404381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0ae290cb1e4382bb164dd260acd759'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:51:31.405080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0ae290cb1e4382bb164dd260acd759'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:51:31.427661 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0ae290cb1e4382bb164dd260acd759'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:51:31.443001 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 606/1184] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:51:31 2022] DELETE /identity/v3/users/5f0ae290cb1e4382bb164dd260acd759 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:51:31.448161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ceaa-1393-409f-8d55-38748c6df3c4 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:31.448614 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ceaa-1393-409f-8d55-38748c6df3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:31.448789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ceaa-1393-409f-8d55-38748c6df3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:31.505296 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-5b63ceaa-1393-409f-8d55-38748c6df3c4 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:51:31.506419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5b63ceaa-1393-409f-8d55-38748c6df3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:31.581132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:31.594969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 579/1185] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:51:31.604776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:51:31.606060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:31.708515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59cc5e-6026-4c15-98e9-943f3f6dc0d4 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:51:31.762602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:31.782448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ae25d-f1a2-4aee-89da-39f7e9145446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:51:31.783049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ae25d-f1a2-4aee-89da-39f7e9145446 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:31.783323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ae25d-f1a2-4aee-89da-39f7e9145446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:31.783590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ae25d-f1a2-4aee-89da-39f7e9145446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:31.785223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-dc2ae25d-f1a2-4aee-89da-39f7e9145446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:31.894311 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ae25d-f1a2-4aee-89da-39f7e9145446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:51:31.895748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ae25d-f1a2-4aee-89da-39f7e9145446 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:51:31.914232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 607/1186] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:51:32.268880 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63ceaa-1393-409f-8d55-38748c6df3c4 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:51:32.270184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5b63ceaa-1393-409f-8d55-38748c6df3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:51:32.282761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63ceaa-1393-409f-8d55-38748c6df3c4 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:51:32.380998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:51:32.381638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:51:32.381916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:51:32.382183 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] PATH_INFO: `/v3/projects/c8b10aa4ce6d475395f22df81f9b0da0` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:51:32.393662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8b10aa4ce6d475395f22df81f9b0da0)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:51:32.394689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: WARNING py.warnings [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:51:32.394689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: warnings.warn(msg) Oct 14 17:51:32.394689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: Oct 14 17:51:32.395919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:51:32.453739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8b10aa4ce6d475395f22df81f9b0da0', 'request_id': 'req-bc21d66e-280b-4ce7-976e-6b92693d525d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:51:32.459634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8b10aa4ce6d475395f22df81f9b0da0', 'request_id': 'req-bc21d66e-280b-4ce7-976e-6b92693d525d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:51:32.470155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8b10aa4ce6d475395f22df81f9b0da0', 'request_id': 'req-bc21d66e-280b-4ce7-976e-6b92693d525d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:51:32.501005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] The token cache is being invalidate because project c8b10aa4ce6d475395f22df81f9b0da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:51:32.501288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:51:32.501710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bc21d66e-280b-4ce7-976e-6b92693d525d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:51:32.503096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 580/1187] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:51:32 2022] DELETE /identity/v3/projects/c8b10aa4ce6d475395f22df81f9b0da0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:51:41.912000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=62 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:51:42.499003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:53:00.204044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899e341-3a20-4af2-aa1f-f3c5eee4e9d2 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:00.204651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899e341-3a20-4af2-aa1f-f3c5eee4e9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:00.204651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899e341-3a20-4af2-aa1f-f3c5eee4e9d2 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:00.205947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 608/1188] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:53:00.210711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8c794-ad09-4c6d-aa6c-4828f17d383c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:00.211111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8c794-ad09-4c6d-aa6c-4828f17d383c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:00.211253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8c794-ad09-4c6d-aa6c-4828f17d383c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:00.292328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-68a8c794-ad09-4c6d-aa6c-4828f17d383c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:53:00.293543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-68a8c794-ad09-4c6d-aa6c-4828f17d383c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:00.395793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 581/1189] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:53:00.551651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899e341-3a20-4af2-aa1f-f3c5eee4e9d2 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:53:00.553797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d899e341-3a20-4af2-aa1f-f3c5eee4e9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:00.676783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899e341-3a20-4af2-aa1f-f3c5eee4e9d2 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:53:00.682868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1633d-30d1-4610-8be8-1d8da06dc64d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:53:00.683467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1633d-30d1-4610-8be8-1d8da06dc64d service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:00.683744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1633d-30d1-4610-8be8-1d8da06dc64d service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:00.684058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1633d-30d1-4610-8be8-1d8da06dc64d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:00.692247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1633d-30d1-4610-8be8-1d8da06dc64d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:53:00.693851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1633d-30d1-4610-8be8-1d8da06dc64d service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:53:00.703777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 609/1190] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:53:01.789803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd74252-d536-48f6-90ee-7e70f8700d67 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:01.790483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd74252-d536-48f6-90ee-7e70f8700d67 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:01.790483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd74252-d536-48f6-90ee-7e70f8700d67 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:01.791980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 582/1191] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:53:01.798793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad704-a24d-42c3-9b0c-0622b27c8b7f None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:01.799259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad704-a24d-42c3-9b0c-0622b27c8b7f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:01.799362 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad704-a24d-42c3-9b0c-0622b27c8b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:01.818497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4aaad704-a24d-42c3-9b0c-0622b27c8b7f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:53:01.819696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4aaad704-a24d-42c3-9b0c-0622b27c8b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:01.828114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 610/1192] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:53:01.982691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd74252-d536-48f6-90ee-7e70f8700d67 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:53:01.997817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd74252-d536-48f6-90ee-7e70f8700d67 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:53:02.003121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723c5cf-f7f2-416c-b957-1999f338d1c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:53:02.003734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723c5cf-f7f2-416c-b957-1999f338d1c9 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:02.004044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723c5cf-f7f2-416c-b957-1999f338d1c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:02.004315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723c5cf-f7f2-416c-b957-1999f338d1c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:02.012396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723c5cf-f7f2-416c-b957-1999f338d1c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:53:02.013961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723c5cf-f7f2-416c-b957-1999f338d1c9 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:53:02.023086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 583/1193] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:53:02.383995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08d237-4f5c-4e7b-9e01-28b0603a8f3c None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:02.384279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08d237-4f5c-4e7b-9e01-28b0603a8f3c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:02.384578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08d237-4f5c-4e7b-9e01-28b0603a8f3c None None] PATH_INFO: `/v3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:02.385903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 611/1194] 10.222.0.25 () {54 vars in 925 bytes} [Fri Oct 14 17:53:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:53:02.398638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d63b8-3d5d-4e45-b441-1130e1c1cea7 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:02.398920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d63b8-3d5d-4e45-b441-1130e1c1cea7 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:02.399189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d63b8-3d5d-4e45-b441-1130e1c1cea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:02.462935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-382d63b8-3d5d-4e45-b441-1130e1c1cea7 None None] MFA Rules not processed for user `73089bc9f85a405b86c4cd3f8cfa830a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:53:02.464185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-382d63b8-3d5d-4e45-b441-1130e1c1cea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:02.480620 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:53:02.535213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 584/1195] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Oct 14 17:53:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:53:06.190855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97431a-f0e2-449b-985b-a6b5a99aa895 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:06.191286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97431a-f0e2-449b-985b-a6b5a99aa895 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:06.191393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97431a-f0e2-449b-985b-a6b5a99aa895 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:06.192720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 612/1196] 10.222.0.27 () {54 vars in 961 bytes} [Fri Oct 14 17:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:53:06.198547 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af428149-8a49-4b27-8bdd-8095df395b0e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:06.198971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af428149-8a49-4b27-8bdd-8095df395b0e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:06.199126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af428149-8a49-4b27-8bdd-8095df395b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:06.218936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-af428149-8a49-4b27-8bdd-8095df395b0e None None] MFA Rules not processed for user `73089bc9f85a405b86c4cd3f8cfa830a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:53:06.220168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-af428149-8a49-4b27-8bdd-8095df395b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:06.229106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 585/1197] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Oct 14 17:53:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:53:06.236604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97431a-f0e2-449b-985b-a6b5a99aa895 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:53:06.251051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97431a-f0e2-449b-985b-a6b5a99aa895 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:53:06.256359 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9828d-4e49-43bf-aabe-0683ebc2fa34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:53:06.256985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9828d-4e49-43bf-aabe-0683ebc2fa34 service placement] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:06.257262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9828d-4e49-43bf-aabe-0683ebc2fa34 service placement] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:06.257758 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9828d-4e49-43bf-aabe-0683ebc2fa34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:06.266862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9828d-4e49-43bf-aabe-0683ebc2fa34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:53:06.268511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9828d-4e49-43bf-aabe-0683ebc2fa34 service placement] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:53:06.278718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 613/1198] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:53:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:53:13.489823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26abb7-2efc-4b3b-9275-a52403e6246b None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:13.490362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26abb7-2efc-4b3b-9275-a52403e6246b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:13.490362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26abb7-2efc-4b3b-9275-a52403e6246b None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:13.491592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 586/1199] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:53:13.498377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f3779-b673-4996-83a7-9092ae5dbeb9 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:13.498835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f3779-b673-4996-83a7-9092ae5dbeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:13.498940 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f3779-b673-4996-83a7-9092ae5dbeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:13.518295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f12f3779-b673-4996-83a7-9092ae5dbeb9 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:53:13.519481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f12f3779-b673-4996-83a7-9092ae5dbeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:13.528850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 614/1200] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:53:13.683949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26abb7-2efc-4b3b-9275-a52403e6246b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:53:13.698698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26abb7-2efc-4b3b-9275-a52403e6246b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:53:13.704116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c2d62-c8b6-44fb-9b67-dc1133150e7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:53:13.704704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c2d62-c8b6-44fb-9b67-dc1133150e7c service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:13.704981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c2d62-c8b6-44fb-9b67-dc1133150e7c service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:13.705250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c2d62-c8b6-44fb-9b67-dc1133150e7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:13.713824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c2d62-c8b6-44fb-9b67-dc1133150e7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:53:13.715381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c2d62-c8b6-44fb-9b67-dc1133150e7c service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:53:13.724681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 587/1201] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:53:14.794733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3e22f-98bd-4ebf-81ca-ede45cbf85a0 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:14.795275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3e22f-98bd-4ebf-81ca-ede45cbf85a0 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:14.795275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3e22f-98bd-4ebf-81ca-ede45cbf85a0 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:14.796691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 615/1202] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 17:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:53:14.802914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7919ca-2073-419a-b8b2-4c018af27f7d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:14.803327 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7919ca-2073-419a-b8b2-4c018af27f7d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:14.803483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7919ca-2073-419a-b8b2-4c018af27f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:14.823250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-cf7919ca-2073-419a-b8b2-4c018af27f7d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:53:14.824470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-cf7919ca-2073-419a-b8b2-4c018af27f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:14.832270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 588/1203] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 17:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:53:14.985064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3e22f-98bd-4ebf-81ca-ede45cbf85a0 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:53:14.999692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3e22f-98bd-4ebf-81ca-ede45cbf85a0 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:53:15.005092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2891920-139d-42ca-8ab3-1573f8aeab6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:53:15.005674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2891920-139d-42ca-8ab3-1573f8aeab6c service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:15.006109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2891920-139d-42ca-8ab3-1573f8aeab6c service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:15.006224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2891920-139d-42ca-8ab3-1573f8aeab6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:15.014297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2891920-139d-42ca-8ab3-1573f8aeab6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:53:15.015866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2891920-139d-42ca-8ab3-1573f8aeab6c service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:53:15.025870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 616/1204] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:53:19.012010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a597d-0f01-4d84-97c8-9c9f11662c67 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:19.012592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a597d-0f01-4d84-97c8-9c9f11662c67 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:19.012592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a597d-0f01-4d84-97c8-9c9f11662c67 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:19.013889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 589/1205] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:53:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:53:19.019425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a263b4a-78fe-418d-ae84-ffd9740d08ef None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:19.019849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a263b4a-78fe-418d-ae84-ffd9740d08ef None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:19.020041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a263b4a-78fe-418d-ae84-ffd9740d08ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:19.041134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8a263b4a-78fe-418d-ae84-ffd9740d08ef None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:53:19.042329 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8a263b4a-78fe-418d-ae84-ffd9740d08ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:19.050535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 617/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:53:19.337777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a597d-0f01-4d84-97c8-9c9f11662c67 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:53:19.339480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-511a597d-0f01-4d84-97c8-9c9f11662c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:19.417668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a597d-0f01-4d84-97c8-9c9f11662c67 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:53:19.421304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605727b-9d6c-4ce9-9d8a-94060a934f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:53:19.421902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605727b-9d6c-4ce9-9d8a-94060a934f0e service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:19.422181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605727b-9d6c-4ce9-9d8a-94060a934f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:19.422454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605727b-9d6c-4ce9-9d8a-94060a934f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:19.431927 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605727b-9d6c-4ce9-9d8a-94060a934f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:53:19.433395 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605727b-9d6c-4ce9-9d8a-94060a934f0e service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:53:19.442834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 590/1207] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:53:21.584600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22109f05-8ea0-4ca4-bbf2-0d55b1ca349a None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:21.585159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22109f05-8ea0-4ca4-bbf2-0d55b1ca349a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:21.585159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22109f05-8ea0-4ca4-bbf2-0d55b1ca349a None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:21.586535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 618/1208] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:53:21.593241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82055ffa-7911-40ef-989e-50bfc1912d65 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:21.593690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82055ffa-7911-40ef-989e-50bfc1912d65 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:21.593795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82055ffa-7911-40ef-989e-50bfc1912d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:21.613843 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-82055ffa-7911-40ef-989e-50bfc1912d65 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:53:21.614946 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-82055ffa-7911-40ef-989e-50bfc1912d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:21.622707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 591/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:53:21.914514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22109f05-8ea0-4ca4-bbf2-0d55b1ca349a None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:53:21.920497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=75 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:53:21.926903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22109f05-8ea0-4ca4-bbf2-0d55b1ca349a None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:53:21.930425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaa984-9616-41b4-be8a-def62b7793f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:53:21.931018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa984-9616-41b4-be8a-def62b7793f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:21.931291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa984-9616-41b4-be8a-def62b7793f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:21.931563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa984-9616-41b4-be8a-def62b7793f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:21.940435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaa984-9616-41b4-be8a-def62b7793f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:53:21.941873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaa984-9616-41b4-be8a-def62b7793f5 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:53:21.951618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 619/1210] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:53:25.724128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3339-4e93-4062-b229-bff04432ba44 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:25.724651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3339-4e93-4062-b229-bff04432ba44 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:25.724651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3339-4e93-4062-b229-bff04432ba44 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:25.725911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 592/1211] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:53:25.731700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d12937-69a0-4705-bae9-0b2f5b806afd None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:25.732134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d12937-69a0-4705-bae9-0b2f5b806afd None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:25.732275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d12937-69a0-4705-bae9-0b2f5b806afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:25.752555 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-d0d12937-69a0-4705-bae9-0b2f5b806afd None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:53:25.753723 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d0d12937-69a0-4705-bae9-0b2f5b806afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:25.762392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 620/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:53:26.073463 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f3339-4e93-4062-b229-bff04432ba44 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:53:26.075153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-460f3339-4e93-4062-b229-bff04432ba44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:26.140004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:53:26.153456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f3339-4e93-4062-b229-bff04432ba44 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:53:26.157402 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689db607-8c22-46c9-b950-578324dae7e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:53:26.158027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689db607-8c22-46c9-b950-578324dae7e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:26.158316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689db607-8c22-46c9-b950-578324dae7e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:26.158581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689db607-8c22-46c9-b950-578324dae7e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:26.167549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689db607-8c22-46c9-b950-578324dae7e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:53:26.169000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689db607-8c22-46c9-b950-578324dae7e1 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:53:26.178146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 593/1213] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:53:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:53:28.313583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68856e-bfb8-4169-874d-984148cfee9a None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:28.314157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68856e-bfb8-4169-874d-984148cfee9a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:28.314157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68856e-bfb8-4169-874d-984148cfee9a None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:28.315511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 621/1214] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 17:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:53:28.320796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1f381-c362-4aaf-8e89-7e20c2e76d5c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:28.322309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1f381-c362-4aaf-8e89-7e20c2e76d5c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:28.322309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1f381-c362-4aaf-8e89-7e20c2e76d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:28.340952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-14b1f381-c362-4aaf-8e89-7e20c2e76d5c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:53:28.342122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-14b1f381-c362-4aaf-8e89-7e20c2e76d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:28.350206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 594/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 17:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:53:28.653567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68856e-bfb8-4169-874d-984148cfee9a None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:53:28.655497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cb68856e-bfb8-4169-874d-984148cfee9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:53:28.669304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68856e-bfb8-4169-874d-984148cfee9a None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:53:28.672810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b3949-f851-4f05-8c82-fb9b32d5ae64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:53:28.673392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3949-f851-4f05-8c82-fb9b32d5ae64 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:53:28.673665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3949-f851-4f05-8c82-fb9b32d5ae64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:53:28.673929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3949-f851-4f05-8c82-fb9b32d5ae64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:53:28.683515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b3949-f851-4f05-8c82-fb9b32d5ae64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:53:28.684977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b3949-f851-4f05-8c82-fb9b32d5ae64 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:53:28.694374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 622/1216] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:53:38.335581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=11 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:53:38.690307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=14 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:54.579845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46d308-15f2-4bac-8f55-e5e649b68c09 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:54.580541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46d308-15f2-4bac-8f55-e5e649b68c09 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:54.580541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46d308-15f2-4bac-8f55-e5e649b68c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:54.610272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-0c46d308-15f2-4bac-8f55-e5e649b68c09 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:54.611432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0c46d308-15f2-4bac-8f55-e5e649b68c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:54.620996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 595/1217] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:54.632545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e466ac-a141-4b09-a0a3-2994832b7d12 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:54.633777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e466ac-a141-4b09-a0a3-2994832b7d12 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:54.634026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e466ac-a141-4b09-a0a3-2994832b7d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:54.661003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-83e466ac-a141-4b09-a0a3-2994832b7d12 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:54.662249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-83e466ac-a141-4b09-a0a3-2994832b7d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:54.674808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 623/1218] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:54.680932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46d308-15f2-4bac-8f55-e5e649b68c09 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:54.689623 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848a967-f49e-4b9e-9d90-940d6cd4f103 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:54.689623 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848a967-f49e-4b9e-9d90-940d6cd4f103 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:54.690017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848a967-f49e-4b9e-9d90-940d6cd4f103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:54.693389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46d308-15f2-4bac-8f55-e5e649b68c09 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:54.702816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95b535-648c-4cbe-9441-418111d12b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:54.703461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95b535-648c-4cbe-9441-418111d12b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:54.703741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95b535-648c-4cbe-9441-418111d12b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:54.704037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95b535-648c-4cbe-9441-418111d12b8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:54.704697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95b535-648c-4cbe-9441-418111d12b8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:54:54.704996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95b535-648c-4cbe-9441-418111d12b8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:54.706893 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95b535-648c-4cbe-9441-418111d12b8f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:54.719819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-5848a967-f49e-4b9e-9d90-940d6cd4f103 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:54.721072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5848a967-f49e-4b9e-9d90-940d6cd4f103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:54.725932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 596/1219] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:54:54.735025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95b535-648c-4cbe-9441-418111d12b8f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:54.740509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 624/1220] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:54.747811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95b535-648c-4cbe-9441-418111d12b8f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:54.750958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e278a1-84de-47f8-a72a-f5c64b1694e8 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:54.751236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e278a1-84de-47f8-a72a-f5c64b1694e8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:54.751491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e278a1-84de-47f8-a72a-f5c64b1694e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:54.761121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811568a3-e4dc-42ed-86d4-cd2ad0cdf6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:54.761759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811568a3-e4dc-42ed-86d4-cd2ad0cdf6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:54.762035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811568a3-e4dc-42ed-86d4-cd2ad0cdf6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:54.762308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811568a3-e4dc-42ed-86d4-cd2ad0cdf6d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:54.762951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811568a3-e4dc-42ed-86d4-cd2ad0cdf6d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:54.764041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: WARNING py.warnings [None req-811568a3-e4dc-42ed-86d4-cd2ad0cdf6d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:54:54.764041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: warnings.warn(msg) Oct 14 17:54:54.764041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: Oct 14 17:54:54.765394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811568a3-e4dc-42ed-86d4-cd2ad0cdf6d6 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:54.782143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-a5e278a1-84de-47f8-a72a-f5c64b1694e8 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:54.783375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a5e278a1-84de-47f8-a72a-f5c64b1694e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:54.803951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 625/1221] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:54:54.809185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e278a1-84de-47f8-a72a-f5c64b1694e8 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:54.826058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e278a1-84de-47f8-a72a-f5c64b1694e8 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:54.838303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955c19f-b357-49cc-9d03-85609a4aae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:54.838924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955c19f-b357-49cc-9d03-85609a4aae57 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:54.839195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955c19f-b357-49cc-9d03-85609a4aae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:54.839464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955c19f-b357-49cc-9d03-85609a4aae57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:54.840473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955c19f-b357-49cc-9d03-85609a4aae57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:54:54.840850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955c19f-b357-49cc-9d03-85609a4aae57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:54.841809 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WARNING py.warnings [None req-e955c19f-b357-49cc-9d03-85609a4aae57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:54:54.841809 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: warnings.warn(msg) Oct 14 17:54:54.841809 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: Oct 14 17:54:54.842693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955c19f-b357-49cc-9d03-85609a4aae57 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:54.850815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 597/1222] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:54 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:54:54.856056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811568a3-e4dc-42ed-86d4-cd2ad0cdf6d6 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:54.860788 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 626/1223] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:54:54.868966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955c19f-b357-49cc-9d03-85609a4aae57 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:54.882167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955c19f-b357-49cc-9d03-85609a4aae57 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.002610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f09fcb-2a2e-40c2-9307-995720eabee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.003245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f09fcb-2a2e-40c2-9307-995720eabee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.003503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f09fcb-2a2e-40c2-9307-995720eabee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.003769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f09fcb-2a2e-40c2-9307-995720eabee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.004435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f09fcb-2a2e-40c2-9307-995720eabee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.006677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f09fcb-2a2e-40c2-9307-995720eabee8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.014293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811568a3-e4dc-42ed-86d4-cd2ad0cdf6d6 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.023404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bf5d4-6537-46e0-9a15-9103d6e72c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.024086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bf5d4-6537-46e0-9a15-9103d6e72c3a admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.024562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bf5d4-6537-46e0-9a15-9103d6e72c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.024838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bf5d4-6537-46e0-9a15-9103d6e72c3a admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.025487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61bf5d4-6537-46e0-9a15-9103d6e72c3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.027423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61bf5d4-6537-46e0-9a15-9103d6e72c3a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.056067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 598/1225] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:54 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:55.056307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 627/1225] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:54 2022] POST /identity/v3/projects => generated 403 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:54:55.057808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc4d32-bab7-43bf-987b-1c16950f2551 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.058319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc4d32-bab7-43bf-987b-1c16950f2551 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.058814 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc4d32-bab7-43bf-987b-1c16950f2551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.060881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90edc8-41e6-4106-812d-86fde84658ba None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.061177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90edc8-41e6-4106-812d-86fde84658ba None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.061510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90edc8-41e6-4106-812d-86fde84658ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.079436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6adc4d32-bab7-43bf-987b-1c16950f2551 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.080717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6adc4d32-bab7-43bf-987b-1c16950f2551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.084814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2e90edc8-41e6-4106-812d-86fde84658ba None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.086970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2e90edc8-41e6-4106-812d-86fde84658ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.142803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 599/1226] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.143813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a6225-1321-4543-9d94-f05fdfc005ac None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.144128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a6225-1321-4543-9d94-f05fdfc005ac None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.144407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a6225-1321-4543-9d94-f05fdfc005ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.152445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 628/1227] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.153700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337758e-a012-44bf-b4cf-733ff649e508 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.154226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337758e-a012-44bf-b4cf-733ff649e508 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.154508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337758e-a012-44bf-b4cf-733ff649e508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.169509 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ac1a6225-1321-4543-9d94-f05fdfc005ac None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.170725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ac1a6225-1321-4543-9d94-f05fdfc005ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.175840 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e337758e-a012-44bf-b4cf-733ff649e508 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.177035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e337758e-a012-44bf-b4cf-733ff649e508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.180309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 600/1228] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.180734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df6a39-5a0a-4b3c-bb8a-74a466b84e8c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.181091 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df6a39-5a0a-4b3c-bb8a-74a466b84e8c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.181377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df6a39-5a0a-4b3c-bb8a-74a466b84e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.192017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 629/1229] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.192734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337758e-a012-44bf-b4cf-733ff649e508 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.203226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c6df6a39-5a0a-4b3c-bb8a-74a466b84e8c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.204460 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c6df6a39-5a0a-4b3c-bb8a-74a466b84e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.204679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337758e-a012-44bf-b4cf-733ff649e508 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.209293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149beb8-a808-45c4-96d0-89eaf7d7873f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.209886 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149beb8-a808-45c4-96d0-89eaf7d7873f admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.210159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149beb8-a808-45c4-96d0-89eaf7d7873f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.210428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149beb8-a808-45c4-96d0-89eaf7d7873f admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.211515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149beb8-a808-45c4-96d0-89eaf7d7873f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.212553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 601/1230] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.213188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149beb8-a808-45c4-96d0-89eaf7d7873f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.213320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df6a39-5a0a-4b3c-bb8a-74a466b84e8c None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.222788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df6a39-5a0a-4b3c-bb8a-74a466b84e8c None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.226468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 630/1231] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:55.226468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613934d-cb07-413b-975e-7927c586127a None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.226468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613934d-cb07-413b-975e-7927c586127a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.226468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613934d-cb07-413b-975e-7927c586127a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.230630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f55214-4948-4ccb-aa71-7646529e6dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.231233 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f55214-4948-4ccb-aa71-7646529e6dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.231509 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f55214-4948-4ccb-aa71-7646529e6dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.231778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f55214-4948-4ccb-aa71-7646529e6dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.232433 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f55214-4948-4ccb-aa71-7646529e6dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.234341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f55214-4948-4ccb-aa71-7646529e6dd9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.245553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f613934d-cb07-413b-975e-7927c586127a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.246750 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f613934d-cb07-413b-975e-7927c586127a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.255410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 631/1232] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.262389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 602/1233] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/users => generated 302 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:55.262389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ae5e5-1cfb-4fd2-9bc5-6793b1444b8a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.262389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ae5e5-1cfb-4fd2-9bc5-6793b1444b8a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.262389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ae5e5-1cfb-4fd2-9bc5-6793b1444b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.265387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc5f19-b20e-49ce-aecd-9c3448ab208d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.265727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc5f19-b20e-49ce-aecd-9c3448ab208d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.265943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc5f19-b20e-49ce-aecd-9c3448ab208d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.283074 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-e31ae5e5-1cfb-4fd2-9bc5-6793b1444b8a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.284319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e31ae5e5-1cfb-4fd2-9bc5-6793b1444b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.286056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7ffc5f19-b20e-49ce-aecd-9c3448ab208d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.287213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7ffc5f19-b20e-49ce-aecd-9c3448ab208d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.293011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 603/1234] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.294255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd739a-c2f5-4364-b826-f9c13ebc110a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.294539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd739a-c2f5-4364-b826-f9c13ebc110a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.294806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd739a-c2f5-4364-b826-f9c13ebc110a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.297926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 632/1235] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.299112 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6641-0e93-410a-8843-edbb86571a76 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.299391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6641-0e93-410a-8843-edbb86571a76 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.299660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6641-0e93-410a-8843-edbb86571a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.302965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:55.316018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-cfdd739a-c2f5-4364-b826-f9c13ebc110a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.317213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-cfdd739a-c2f5-4364-b826-f9c13ebc110a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.318977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7a7f6641-0e93-410a-8843-edbb86571a76 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.320159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7a7f6641-0e93-410a-8843-edbb86571a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.325052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 604/1236] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.326282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4fa15-7e89-40d7-b7cb-00352ff890e8 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.326565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4fa15-7e89-40d7-b7cb-00352ff890e8 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.326833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4fa15-7e89-40d7-b7cb-00352ff890e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.330334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 633/1237] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.330975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f6641-0e93-410a-8843-edbb86571a76 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.341288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f6641-0e93-410a-8843-edbb86571a76 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.347581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e82560-34a0-4442-b9eb-8015f746f570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.347581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e82560-34a0-4442-b9eb-8015f746f570 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.347993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e82560-34a0-4442-b9eb-8015f746f570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.348096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e82560-34a0-4442-b9eb-8015f746f570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.349309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e82560-34a0-4442-b9eb-8015f746f570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.349438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-06c4fa15-7e89-40d7-b7cb-00352ff890e8 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.350320 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e82560-34a0-4442-b9eb-8015f746f570 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.350439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-06c4fa15-7e89-40d7-b7cb-00352ff890e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.358987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 605/1238] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.359777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4fa15-7e89-40d7-b7cb-00352ff890e8 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.362196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 634/1239] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:55.363398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd30c98-4c8f-401f-aa78-6ff4312e177e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.363682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd30c98-4c8f-401f-aa78-6ff4312e177e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.363978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd30c98-4c8f-401f-aa78-6ff4312e177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.371347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4fa15-7e89-40d7-b7cb-00352ff890e8 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.373484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:55.377360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb118320-18db-4cc2-9f19-3ff250a6f91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.378060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb118320-18db-4cc2-9f19-3ff250a6f91b admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.378406 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb118320-18db-4cc2-9f19-3ff250a6f91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.378746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb118320-18db-4cc2-9f19-3ff250a6f91b admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.379465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb118320-18db-4cc2-9f19-3ff250a6f91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.381349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb118320-18db-4cc2-9f19-3ff250a6f91b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.385799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-fbd30c98-4c8f-401f-aa78-6ff4312e177e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.387037 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-fbd30c98-4c8f-401f-aa78-6ff4312e177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.397508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 606/1240] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:55.397672 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 635/1241] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.397795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb118320-18db-4cc2-9f19-3ff250a6f91b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.397899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd30c98-4c8f-401f-aa78-6ff4312e177e None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.409066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd30c98-4c8f-401f-aa78-6ff4312e177e None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.409654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb118320-18db-4cc2-9f19-3ff250a6f91b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.413684 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c0e94-037c-4d94-b9c1-866642d1be6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.414969 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c0e94-037c-4d94-b9c1-866642d1be6f admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.415764 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58856d4-f26c-443e-9914-635ab565e638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.415906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c0e94-037c-4d94-b9c1-866642d1be6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.415906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c0e94-037c-4d94-b9c1-866642d1be6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.416088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58856d4-f26c-443e-9914-635ab565e638 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.416181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58856d4-f26c-443e-9914-635ab565e638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.416256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c0e94-037c-4d94-b9c1-866642d1be6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:54:55.416419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58856d4-f26c-443e-9914-635ab565e638 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.416542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c0e94-037c-4d94-b9c1-866642d1be6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.417054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58856d4-f26c-443e-9914-635ab565e638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:54:55.417353 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58856d4-f26c-443e-9914-635ab565e638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.418028 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c0e94-037c-4d94-b9c1-866642d1be6f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.418960 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58856d4-f26c-443e-9914-635ab565e638 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.429921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 636/1242] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:54:55.430984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c0e94-037c-4d94-b9c1-866642d1be6f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.431213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 607/1243] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:54:55.432089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58856d4-f26c-443e-9914-635ab565e638 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.443130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58856d4-f26c-443e-9914-635ab565e638 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.447301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c0e94-037c-4d94-b9c1-866642d1be6f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.447817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b33ae5-33fa-462b-ac7f-55e21bb51a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.448440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b33ae5-33fa-462b-ac7f-55e21bb51a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.448710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b33ae5-33fa-462b-ac7f-55e21bb51a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.448980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b33ae5-33fa-462b-ac7f-55e21bb51a2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.449603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b33ae5-33fa-462b-ac7f-55e21bb51a2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:54:55.449900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b33ae5-33fa-462b-ac7f-55e21bb51a2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.451480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b33ae5-33fa-462b-ac7f-55e21bb51a2e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.451831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25112407-41cf-4bfe-ae3c-763d4e6eba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.452450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25112407-41cf-4bfe-ae3c-763d4e6eba89 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.452778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25112407-41cf-4bfe-ae3c-763d4e6eba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.453050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25112407-41cf-4bfe-ae3c-763d4e6eba89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.453680 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25112407-41cf-4bfe-ae3c-763d4e6eba89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:54:55.454008 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25112407-41cf-4bfe-ae3c-763d4e6eba89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.455585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25112407-41cf-4bfe-ae3c-763d4e6eba89 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.463651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 608/1244] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:54:55.464552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b33ae5-33fa-462b-ac7f-55e21bb51a2e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.475405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 637/1245] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:54:55.475405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25112407-41cf-4bfe-ae3c-763d4e6eba89 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.479281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b33ae5-33fa-462b-ac7f-55e21bb51a2e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.481844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25112407-41cf-4bfe-ae3c-763d4e6eba89 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.483832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415bc0b-6ce1-4ee2-a347-5751a5b02a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.484497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415bc0b-6ce1-4ee2-a347-5751a5b02a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.484799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415bc0b-6ce1-4ee2-a347-5751a5b02a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.485072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415bc0b-6ce1-4ee2-a347-5751a5b02a73 admin admin] PATH_INFO: `/v3/projects/f5488304972c48fab9a2b9711cf7e9c8/users/762e1e38e4234803991187db9b870b9c/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.486674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dcde1-e51c-4299-949b-a38b0a3a7ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.487260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dcde1-e51c-4299-949b-a38b0a3a7ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.487537 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dcde1-e51c-4299-949b-a38b0a3a7ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.487806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dcde1-e51c-4299-949b-a38b0a3a7ba1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.488472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805dcde1-e51c-4299-949b-a38b0a3a7ba1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:54:55.488772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805dcde1-e51c-4299-949b-a38b0a3a7ba1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.490356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805dcde1-e51c-4299-949b-a38b0a3a7ba1 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.499981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415bc0b-6ce1-4ee2-a347-5751a5b02a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5488304972c48fab9a2b9711cf7e9c8, user_id=762e1e38e4234803991187db9b870b9c, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.502049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 638/1246] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:54:55.502802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415bc0b-6ce1-4ee2-a347-5751a5b02a73 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.502910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dcde1-e51c-4299-949b-a38b0a3a7ba1 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.513239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dcde1-e51c-4299-949b-a38b0a3a7ba1 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.517372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 609/1247] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:55 2022] PUT /identity/v3/projects/f5488304972c48fab9a2b9711cf7e9c8/users/762e1e38e4234803991187db9b870b9c/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:55.518406 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415bc0b-6ce1-4ee2-a347-5751a5b02a73 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.572997 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095a9e6-248f-4dd7-93ec-ea8120776228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.573604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095a9e6-248f-4dd7-93ec-ea8120776228 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.573880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095a9e6-248f-4dd7-93ec-ea8120776228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.574150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095a9e6-248f-4dd7-93ec-ea8120776228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.574793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095a9e6-248f-4dd7-93ec-ea8120776228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.576624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095a9e6-248f-4dd7-93ec-ea8120776228 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.582727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415bc0b-6ce1-4ee2-a347-5751a5b02a73 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.589931 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 639/1248] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:55.589931 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095a9e6-248f-4dd7-93ec-ea8120776228 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.590159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d7524-a9c4-4632-8ded-c26c53b25819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.590159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d7524-a9c4-4632-8ded-c26c53b25819 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.590159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d7524-a9c4-4632-8ded-c26c53b25819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.590159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d7524-a9c4-4632-8ded-c26c53b25819 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.590551 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d7524-a9c4-4632-8ded-c26c53b25819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:54:55.590551 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d7524-a9c4-4632-8ded-c26c53b25819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.591928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d7524-a9c4-4632-8ded-c26c53b25819 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.600432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095a9e6-248f-4dd7-93ec-ea8120776228 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.611666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 610/1249] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:54:55.611666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d7524-a9c4-4632-8ded-c26c53b25819 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.611955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a509b1-f845-4c50-a8e8-469cfd8120ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.611955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a509b1-f845-4c50-a8e8-469cfd8120ab admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.611955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a509b1-f845-4c50-a8e8-469cfd8120ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.611955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a509b1-f845-4c50-a8e8-469cfd8120ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.612327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a509b1-f845-4c50-a8e8-469cfd8120ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.612414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a509b1-f845-4c50-a8e8-469cfd8120ab admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.615245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d7524-a9c4-4632-8ded-c26c53b25819 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.619927 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190d4a8-cf78-410d-9c35-274372834a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.620512 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190d4a8-cf78-410d-9c35-274372834a17 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.620789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190d4a8-cf78-410d-9c35-274372834a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.621130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190d4a8-cf78-410d-9c35-274372834a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.621702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190d4a8-cf78-410d-9c35-274372834a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.623562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190d4a8-cf78-410d-9c35-274372834a17 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.640424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 640/1250] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/projects => generated 401 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:55.641227 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a509b1-f845-4c50-a8e8-469cfd8120ab admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.659000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 611/1251] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:55.659000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190d4a8-cf78-410d-9c35-274372834a17 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.709196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a509b1-f845-4c50-a8e8-469cfd8120ab admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.719378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190d4a8-cf78-410d-9c35-274372834a17 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.724053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cecbef-85d6-439c-9902-2d96b980108b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.724652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cecbef-85d6-439c-9902-2d96b980108b admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.724929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cecbef-85d6-439c-9902-2d96b980108b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.725198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cecbef-85d6-439c-9902-2d96b980108b admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.725817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cecbef-85d6-439c-9902-2d96b980108b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.728026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cecbef-85d6-439c-9902-2d96b980108b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.732436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:55.749482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:55.758299 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 612/1252] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:55.759186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cecbef-85d6-439c-9902-2d96b980108b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.772775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3455c4-9307-4324-b1bd-c45070dac734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.773499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3455c4-9307-4324-b1bd-c45070dac734 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.774225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3455c4-9307-4324-b1bd-c45070dac734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.775130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3455c4-9307-4324-b1bd-c45070dac734 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.775749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3455c4-9307-4324-b1bd-c45070dac734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.777686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3455c4-9307-4324-b1bd-c45070dac734 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.804522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 641/1253] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:55.806068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44223016-e8fb-4a19-8bf2-0e4f02e9fa8b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.806349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44223016-e8fb-4a19-8bf2-0e4f02e9fa8b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.807067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44223016-e8fb-4a19-8bf2-0e4f02e9fa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.823300 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cecbef-85d6-439c-9902-2d96b980108b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.826141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-44223016-e8fb-4a19-8bf2-0e4f02e9fa8b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:55.827322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-44223016-e8fb-4a19-8bf2-0e4f02e9fa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:55.880517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aeed69-175a-4a79-9f3f-02921f637814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.881124 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aeed69-175a-4a79-9f3f-02921f637814 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.881402 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aeed69-175a-4a79-9f3f-02921f637814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.881671 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aeed69-175a-4a79-9f3f-02921f637814 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.882296 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aeed69-175a-4a79-9f3f-02921f637814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.884881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aeed69-175a-4a79-9f3f-02921f637814 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.888010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 642/1254] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:55.888818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44223016-e8fb-4a19-8bf2-0e4f02e9fa8b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.900045 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44223016-e8fb-4a19-8bf2-0e4f02e9fa8b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.905569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f487e0-5c1b-406a-8396-1412341c4e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:55.906363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f487e0-5c1b-406a-8396-1412341c4e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:55.906916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f487e0-5c1b-406a-8396-1412341c4e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:55.907191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f487e0-5c1b-406a-8396-1412341c4e04 admin admin] PATH_INFO: `/v3/projects/eacbbec0ccbb4ef097d046564d61d468/users/bf96e2a560ae495e89da872a4f676c78/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:55.913105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 613/1255] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:55.913903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aeed69-175a-4a79-9f3f-02921f637814 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.925063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f487e0-5c1b-406a-8396-1412341c4e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eacbbec0ccbb4ef097d046564d61d468, user_id=bf96e2a560ae495e89da872a4f676c78, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:55.927716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f487e0-5c1b-406a-8396-1412341c4e04 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:55.941241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 643/1256] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:55 2022] PUT /identity/v3/projects/eacbbec0ccbb4ef097d046564d61d468/users/bf96e2a560ae495e89da872a4f676c78/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:55.942005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f487e0-5c1b-406a-8396-1412341c4e04 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:55.976810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aeed69-175a-4a79-9f3f-02921f637814 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:55.996513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:56.006344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f487e0-5c1b-406a-8396-1412341c4e04 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.011947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b80ac-8ec0-4e07-aaa6-d9fca800ec2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.012586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b80ac-8ec0-4e07-aaa6-d9fca800ec2e admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.012886 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b80ac-8ec0-4e07-aaa6-d9fca800ec2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.013151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b80ac-8ec0-4e07-aaa6-d9fca800ec2e admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.013779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b80ac-8ec0-4e07-aaa6-d9fca800ec2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.014800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WARNING py.warnings [None req-f58b80ac-8ec0-4e07-aaa6-d9fca800ec2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:54:56.014800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: warnings.warn(msg) Oct 14 17:54:56.014800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: Oct 14 17:54:56.016078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b80ac-8ec0-4e07-aaa6-d9fca800ec2e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.037094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab5056-24fa-4f4f-94b3-27d50c8dc6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.037707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab5056-24fa-4f4f-94b3-27d50c8dc6fa admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.038004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab5056-24fa-4f4f-94b3-27d50c8dc6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.038275 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab5056-24fa-4f4f-94b3-27d50c8dc6fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.038900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab5056-24fa-4f4f-94b3-27d50c8dc6fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.041017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab5056-24fa-4f4f-94b3-27d50c8dc6fa admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.045803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 644/1257] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:56.045803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b80ac-8ec0-4e07-aaa6-d9fca800ec2e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.054429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b80ac-8ec0-4e07-aaa6-d9fca800ec2e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.060887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecde6e-2546-454e-8023-2efd56d1bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.061519 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecde6e-2546-454e-8023-2efd56d1bd39 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.061801 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecde6e-2546-454e-8023-2efd56d1bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.062126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecde6e-2546-454e-8023-2efd56d1bd39 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.062811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecde6e-2546-454e-8023-2efd56d1bd39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.064777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecde6e-2546-454e-8023-2efd56d1bd39 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.069216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 614/1258] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:55 2022] POST /identity/v3/projects => generated 389 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:56.070030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab5056-24fa-4f4f-94b3-27d50c8dc6fa admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.091443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 645/1259] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:56 2022] POST /identity/v3/users => generated 296 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:56.091443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecde6e-2546-454e-8023-2efd56d1bd39 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.098708 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecde6e-2546-454e-8023-2efd56d1bd39 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.108174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:56.138278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab5056-24fa-4f4f-94b3-27d50c8dc6fa admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.142915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85963ee1-43c5-4117-93dd-e5da60a0124f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.143525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85963ee1-43c5-4117-93dd-e5da60a0124f admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.143823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85963ee1-43c5-4117-93dd-e5da60a0124f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.144129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85963ee1-43c5-4117-93dd-e5da60a0124f admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.144753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85963ee1-43c5-4117-93dd-e5da60a0124f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.146698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85963ee1-43c5-4117-93dd-e5da60a0124f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.157797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb541d-6970-4be7-85a6-cf79cfbd64fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.158403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb541d-6970-4be7-85a6-cf79cfbd64fe admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.158683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb541d-6970-4be7-85a6-cf79cfbd64fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.158996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb541d-6970-4be7-85a6-cf79cfbd64fe admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.159796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb541d-6970-4be7-85a6-cf79cfbd64fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.162218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb541d-6970-4be7-85a6-cf79cfbd64fe admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.169448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 615/1260] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:56 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:56.170652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85963ee1-43c5-4117-93dd-e5da60a0124f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.186098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 646/1261] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:56 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:56.186898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb541d-6970-4be7-85a6-cf79cfbd64fe admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.197694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb541d-6970-4be7-85a6-cf79cfbd64fe admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.202782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51e184-5b62-4972-93d6-698a2ca7814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.203963 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51e184-5b62-4972-93d6-698a2ca7814d admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.204451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51e184-5b62-4972-93d6-698a2ca7814d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.204934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51e184-5b62-4972-93d6-698a2ca7814d admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.206111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51e184-5b62-4972-93d6-698a2ca7814d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.209856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51e184-5b62-4972-93d6-698a2ca7814d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.237289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 647/1262] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:56 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:56.237509 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85963ee1-43c5-4117-93dd-e5da60a0124f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.237617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a234c-7f26-4c82-90b5-0b57f0582ed6 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.237617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a234c-7f26-4c82-90b5-0b57f0582ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.237617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a234c-7f26-4c82-90b5-0b57f0582ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.239290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0effb-6044-4bef-ad68-841ecd9095e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.239908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0effb-6044-4bef-ad68-841ecd9095e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.240193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0effb-6044-4bef-ad68-841ecd9095e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.240462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0effb-6044-4bef-ad68-841ecd9095e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.250451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0effb-6044-4bef-ad68-841ecd9095e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.251897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0effb-6044-4bef-ad68-841ecd9095e7 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.256399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8d3a234c-7f26-4c82-90b5-0b57f0582ed6 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:56.257593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8d3a234c-7f26-4c82-90b5-0b57f0582ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:56.262468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 616/1263] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:54:56.263251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0effb-6044-4bef-ad68-841ecd9095e7 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.268847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 648/1264] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:56.268847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a234c-7f26-4c82-90b5-0b57f0582ed6 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.272315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0effb-6044-4bef-ad68-841ecd9095e7 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.278408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d853c2-858f-4dd8-acda-3e0d8f17a1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.279367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d853c2-858f-4dd8-acda-3e0d8f17a1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.279507 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d853c2-858f-4dd8-acda-3e0d8f17a1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.279665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d853c2-858f-4dd8-acda-3e0d8f17a1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.280703 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d853c2-858f-4dd8-acda-3e0d8f17a1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.280824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a234c-7f26-4c82-90b5-0b57f0582ed6 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.282208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d853c2-858f-4dd8-acda-3e0d8f17a1d1 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.286802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce888c2-f79b-4fb8-8369-03870a0d9185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.287856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce888c2-f79b-4fb8-8369-03870a0d9185 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.288499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce888c2-f79b-4fb8-8369-03870a0d9185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.288499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce888c2-f79b-4fb8-8369-03870a0d9185 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.289330 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce888c2-f79b-4fb8-8369-03870a0d9185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.291643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce888c2-f79b-4fb8-8369-03870a0d9185 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.294186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 617/1265] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.295741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d853c2-858f-4dd8-acda-3e0d8f17a1d1 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.309116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d853c2-858f-4dd8-acda-3e0d8f17a1d1 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.313994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978cbac-6f00-408a-bb74-385d3c202484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.314593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978cbac-6f00-408a-bb74-385d3c202484 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.314870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978cbac-6f00-408a-bb74-385d3c202484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.315140 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978cbac-6f00-408a-bb74-385d3c202484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.316031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978cbac-6f00-408a-bb74-385d3c202484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.317632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978cbac-6f00-408a-bb74-385d3c202484 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.318063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 649/1266] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:56 2022] POST /identity/v3/users => generated 295 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:56.318843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce888c2-f79b-4fb8-8369-03870a0d9185 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.329887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 618/1267] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.330642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978cbac-6f00-408a-bb74-385d3c202484 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.331124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce888c2-f79b-4fb8-8369-03870a0d9185 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.337995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85b7b9-ebd3-42d6-9af5-0f16b4a91444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.338768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b7b9-ebd3-42d6-9af5-0f16b4a91444 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.339212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b7b9-ebd3-42d6-9af5-0f16b4a91444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.339467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b7b9-ebd3-42d6-9af5-0f16b4a91444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.340456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85b7b9-ebd3-42d6-9af5-0f16b4a91444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.341026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978cbac-6f00-408a-bb74-385d3c202484 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.342080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85b7b9-ebd3-42d6-9af5-0f16b4a91444 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.349886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec0707-2791-479d-8830-ecfe5df5ed6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.350453 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec0707-2791-479d-8830-ecfe5df5ed6f admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.350788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec0707-2791-479d-8830-ecfe5df5ed6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.351126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec0707-2791-479d-8830-ecfe5df5ed6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.351768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec0707-2791-479d-8830-ecfe5df5ed6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.353563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec0707-2791-479d-8830-ecfe5df5ed6f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.356249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 650/1268] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.357410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85b7b9-ebd3-42d6-9af5-0f16b4a91444 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.365631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 619/1269] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.366452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec0707-2791-479d-8830-ecfe5df5ed6f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.368997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2eec0707-2791-479d-8830-ecfe5df5ed6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:56.376067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85b7b9-ebd3-42d6-9af5-0f16b4a91444 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.381868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec0707-2791-479d-8830-ecfe5df5ed6f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.382013 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35342f12-4191-441a-9ebe-dd2d9dd29014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.383453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35342f12-4191-441a-9ebe-dd2d9dd29014 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.383812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35342f12-4191-441a-9ebe-dd2d9dd29014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.384214 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35342f12-4191-441a-9ebe-dd2d9dd29014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.385286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35342f12-4191-441a-9ebe-dd2d9dd29014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.388042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd9b19-1130-4628-b24a-aa9b81b1f79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.388198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35342f12-4191-441a-9ebe-dd2d9dd29014 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.388313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd9b19-1130-4628-b24a-aa9b81b1f79a service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.388313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd9b19-1130-4628-b24a-aa9b81b1f79a service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.388313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd9b19-1130-4628-b24a-aa9b81b1f79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.395658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:56.398403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd9b19-1130-4628-b24a-aa9b81b1f79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.400432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd9b19-1130-4628-b24a-aa9b81b1f79a service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.403495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 651/1270] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.407735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35342f12-4191-441a-9ebe-dd2d9dd29014 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.415335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 620/1271] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:54:56.416229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd9b19-1130-4628-b24a-aa9b81b1f79a service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.427462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd9b19-1130-4628-b24a-aa9b81b1f79a service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.433167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be0dfb-300e-48fa-8f30-896e2c19f328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.433793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be0dfb-300e-48fa-8f30-896e2c19f328 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.434070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be0dfb-300e-48fa-8f30-896e2c19f328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.434335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be0dfb-300e-48fa-8f30-896e2c19f328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.434989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be0dfb-300e-48fa-8f30-896e2c19f328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.436500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35342f12-4191-441a-9ebe-dd2d9dd29014 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.436901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be0dfb-300e-48fa-8f30-896e2c19f328 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.444020 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07323525-c246-4fe2-928e-347c65483630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.446224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07323525-c246-4fe2-928e-347c65483630 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.446669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07323525-c246-4fe2-928e-347c65483630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.447115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07323525-c246-4fe2-928e-347c65483630 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.448017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07323525-c246-4fe2-928e-347c65483630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.449876 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 621/1272] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.450742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be0dfb-300e-48fa-8f30-896e2c19f328 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.451155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07323525-c246-4fe2-928e-347c65483630 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.463540 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be0dfb-300e-48fa-8f30-896e2c19f328 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.466691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 652/1273] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.467796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07323525-c246-4fe2-928e-347c65483630 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.470185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262d2d3-312a-4ec4-855a-f073c7e59128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.473479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262d2d3-312a-4ec4-855a-f073c7e59128 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.473479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262d2d3-312a-4ec4-855a-f073c7e59128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.473479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262d2d3-312a-4ec4-855a-f073c7e59128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.473479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262d2d3-312a-4ec4-855a-f073c7e59128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.474969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262d2d3-312a-4ec4-855a-f073c7e59128 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.479318 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07323525-c246-4fe2-928e-347c65483630 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.485048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f55096-2b4b-4181-8ddb-7b002f1189ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.487122 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f55096-2b4b-4181-8ddb-7b002f1189ec admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.487550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f55096-2b4b-4181-8ddb-7b002f1189ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.487957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f55096-2b4b-4181-8ddb-7b002f1189ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.488845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f55096-2b4b-4181-8ddb-7b002f1189ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.489630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 622/1274] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.491248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f55096-2b4b-4181-8ddb-7b002f1189ec admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.491610 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262d2d3-312a-4ec4-855a-f073c7e59128 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.503904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262d2d3-312a-4ec4-855a-f073c7e59128 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.505841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 653/1275] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.506667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f55096-2b4b-4181-8ddb-7b002f1189ec admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.511323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3d8bd-26ad-4cb1-b2c4-77c4be1622d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.512241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3d8bd-26ad-4cb1-b2c4-77c4be1622d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.512776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3d8bd-26ad-4cb1-b2c4-77c4be1622d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.513337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3d8bd-26ad-4cb1-b2c4-77c4be1622d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.514424 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3d8bd-26ad-4cb1-b2c4-77c4be1622d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.517330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3d8bd-26ad-4cb1-b2c4-77c4be1622d9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.520756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f55096-2b4b-4181-8ddb-7b002f1189ec admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.527846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6ac2a-7c8b-4419-b6d1-1bef3d937ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.528731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ac2a-7c8b-4419-b6d1-1bef3d937ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.529116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ac2a-7c8b-4419-b6d1-1bef3d937ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.529543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ac2a-7c8b-4419-b6d1-1bef3d937ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.530403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6ac2a-7c8b-4419-b6d1-1bef3d937ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.532934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6ac2a-7c8b-4419-b6d1-1bef3d937ea5 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.534153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 623/1276] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.535273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3d8bd-26ad-4cb1-b2c4-77c4be1622d9 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.548972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3d8bd-26ad-4cb1-b2c4-77c4be1622d9 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.553036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 654/1277] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.554199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6ac2a-7c8b-4419-b6d1-1bef3d937ea5 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.557713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6a74-7703-4dd2-ac1e-a3b90f2ffeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.558484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6a74-7703-4dd2-ac1e-a3b90f2ffeaf admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.558831 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6a74-7703-4dd2-ac1e-a3b90f2ffeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.559166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6a74-7703-4dd2-ac1e-a3b90f2ffeaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.559797 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a6a74-7703-4dd2-ac1e-a3b90f2ffeaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.561959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a6a74-7703-4dd2-ac1e-a3b90f2ffeaf admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.570990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6ac2a-7c8b-4419-b6d1-1bef3d937ea5 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.576248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c05fc0-2aa9-4b1c-ad35-8c772f87869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.577172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c05fc0-2aa9-4b1c-ad35-8c772f87869e admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.577525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c05fc0-2aa9-4b1c-ad35-8c772f87869e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.577869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c05fc0-2aa9-4b1c-ad35-8c772f87869e admin admin] PATH_INFO: `/v3/projects/a0460ab6e3a54a7a9d95da330b625879/users/8dddbcca59ae44ff9aeb7e274d1e2558/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.577986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 624/1278] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:56.579304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6a74-7703-4dd2-ac1e-a3b90f2ffeaf admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.591372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6a74-7703-4dd2-ac1e-a3b90f2ffeaf admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.596477 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c05fc0-2aa9-4b1c-ad35-8c772f87869e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0460ab6e3a54a7a9d95da330b625879, user_id=8dddbcca59ae44ff9aeb7e274d1e2558, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.598431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13d828-e974-4a76-b65b-5c7407608d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.599147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13d828-e974-4a76-b65b-5c7407608d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.599574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13d828-e974-4a76-b65b-5c7407608d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.599872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13d828-e974-4a76-b65b-5c7407608d8d admin admin] PATH_INFO: `/v3/projects/530fdfc3448a40b7aa85170873d3f0ff/users/c091bf027839447da4c713deb291ed1f/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.600181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c05fc0-2aa9-4b1c-ad35-8c772f87869e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.620376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 655/1279] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:56 2022] PUT /identity/v3/projects/a0460ab6e3a54a7a9d95da330b625879/users/8dddbcca59ae44ff9aeb7e274d1e2558/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:56.621536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c05fc0-2aa9-4b1c-ad35-8c772f87869e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.623384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13d828-e974-4a76-b65b-5c7407608d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=530fdfc3448a40b7aa85170873d3f0ff, user_id=c091bf027839447da4c713deb291ed1f, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.627162 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13d828-e974-4a76-b65b-5c7407608d8d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.680330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 625/1280] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:56 2022] PUT /identity/v3/projects/530fdfc3448a40b7aa85170873d3f0ff/users/c091bf027839447da4c713deb291ed1f/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:56.682747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13d828-e974-4a76-b65b-5c7407608d8d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.684313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:56.713064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c05fc0-2aa9-4b1c-ad35-8c772f87869e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.797848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b23d98-4b83-4664-850b-f3cf10d52d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.798694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b23d98-4b83-4664-850b-f3cf10d52d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.798974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b23d98-4b83-4664-850b-f3cf10d52d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.799300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b23d98-4b83-4664-850b-f3cf10d52d7a admin admin] PATH_INFO: `/v3/projects/32dab4af21f74138b614f6a0b0a30f13/users/d680d547ec9043d48ab6cec6dcd08468/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.819381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b23d98-4b83-4664-850b-f3cf10d52d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=32dab4af21f74138b614f6a0b0a30f13, user_id=d680d547ec9043d48ab6cec6dcd08468, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.822332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b23d98-4b83-4664-850b-f3cf10d52d7a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.832715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13d828-e974-4a76-b65b-5c7407608d8d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.849391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 656/1281] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:56 2022] PUT /identity/v3/projects/32dab4af21f74138b614f6a0b0a30f13/users/d680d547ec9043d48ab6cec6dcd08468/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:56.849391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b23d98-4b83-4664-850b-f3cf10d52d7a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:56.921526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b23d98-4b83-4664-850b-f3cf10d52d7a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:56.928348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08882fea-2427-4225-9e03-276ffde2ded3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.929264 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08882fea-2427-4225-9e03-276ffde2ded3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.929610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08882fea-2427-4225-9e03-276ffde2ded3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.929982 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08882fea-2427-4225-9e03-276ffde2ded3 admin admin] PATH_INFO: `/v3/projects/bfe90c27e0e44c4794ccff00eac150ca/users/07441f3b8a3841b0ba3677ec38fe7f32/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.948347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08882fea-2427-4225-9e03-276ffde2ded3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe90c27e0e44c4794ccff00eac150ca, user_id=07441f3b8a3841b0ba3677ec38fe7f32, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:56.951785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08882fea-2427-4225-9e03-276ffde2ded3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:56.978341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751887b-cbb3-4b4d-b37d-df708bfa3527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:56.978341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751887b-cbb3-4b4d-b37d-df708bfa3527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:56.978578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751887b-cbb3-4b4d-b37d-df708bfa3527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:56.978974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751887b-cbb3-4b4d-b37d-df708bfa3527 admin admin] PATH_INFO: `/v3/projects/7c55aced11be4d46890f8d417890c067/users/65d78df9d1e947f9a6cffd2f9e4be22e/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:56.979146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 657/1282] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:56 2022] PUT /identity/v3/projects/bfe90c27e0e44c4794ccff00eac150ca/users/07441f3b8a3841b0ba3677ec38fe7f32/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:56.981219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08882fea-2427-4225-9e03-276ffde2ded3 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:57.002316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751887b-cbb3-4b4d-b37d-df708bfa3527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c55aced11be4d46890f8d417890c067, user_id=65d78df9d1e947f9a6cffd2f9e4be22e, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:57.005999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751887b-cbb3-4b4d-b37d-df708bfa3527 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:57.035164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 626/1283] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:56 2022] PUT /identity/v3/projects/7c55aced11be4d46890f8d417890c067/users/65d78df9d1e947f9a6cffd2f9e4be22e/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:57.049621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddb6d-ef2c-4b16-bd93-45e8c8e041d6 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.049970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddb6d-ef2c-4b16-bd93-45e8c8e041d6 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.050849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddb6d-ef2c-4b16-bd93-45e8c8e041d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.086620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:57.093242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-eafddb6d-ef2c-4b16-bd93-45e8c8e041d6 None None] MFA Rules not processed for user `8dddbcca59ae44ff9aeb7e274d1e2558`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:57.094079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08882fea-2427-4225-9e03-276ffde2ded3 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:57.094647 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-eafddb6d-ef2c-4b16-bd93-45e8c8e041d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:57.100373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:57.177415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8955e78-edba-46ea-b2f6-7dbf0907cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:57.178046 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955e78-edba-46ea-b2f6-7dbf0907cff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.178567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955e78-edba-46ea-b2f6-7dbf0907cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.178681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955e78-edba-46ea-b2f6-7dbf0907cff2 admin admin] PATH_INFO: `/v3/projects/261d62defa42436cbcc5e712c62b76f4/users/b9187955c3cf4b77ba86e67a6f378e6a/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.181350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 627/1284] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:57.184133 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799c757-9a83-46c1-a773-c7300f6c3ce7 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.184426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799c757-9a83-46c1-a773-c7300f6c3ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.184702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799c757-9a83-46c1-a773-c7300f6c3ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.199115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8955e78-edba-46ea-b2f6-7dbf0907cff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=261d62defa42436cbcc5e712c62b76f4, user_id=b9187955c3cf4b77ba86e67a6f378e6a, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:57.207501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8955e78-edba-46ea-b2f6-7dbf0907cff2 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:57.211123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2799c757-9a83-46c1-a773-c7300f6c3ce7 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:57.212857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2799c757-9a83-46c1-a773-c7300f6c3ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:57.221311 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 628/1285] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:57.224733 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a43792-ea4a-4453-88d2-81d4f807ca1e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.224733 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a43792-ea4a-4453-88d2-81d4f807ca1e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.224733 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a43792-ea4a-4453-88d2-81d4f807ca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.235857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 658/1286] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:56 2022] PUT /identity/v3/projects/261d62defa42436cbcc5e712c62b76f4/users/b9187955c3cf4b77ba86e67a6f378e6a/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:57.235857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d6b78-2521-4af4-9bb4-7419afced12b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.235857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d6b78-2521-4af4-9bb4-7419afced12b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.235857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d6b78-2521-4af4-9bb4-7419afced12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.259315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-21a43792-ea4a-4453-88d2-81d4f807ca1e None None] MFA Rules not processed for user `d680d547ec9043d48ab6cec6dcd08468`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:57.261505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-21a43792-ea4a-4453-88d2-81d4f807ca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:57.262919 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c83d6b78-2521-4af4-9bb4-7419afced12b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:57.264326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c83d6b78-2521-4af4-9bb4-7419afced12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:57.355398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 629/1287] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:57.356773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9b919-123f-44d9-8171-26695d000724 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.357059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9b919-123f-44d9-8171-26695d000724 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.357328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9b919-123f-44d9-8171-26695d000724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.395778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 659/1288] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:57.396759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d6b78-2521-4af4-9bb4-7419afced12b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:57.415853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d6b78-2521-4af4-9bb4-7419afced12b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:57.417736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-78a9b919-123f-44d9-8171-26695d000724 None None] MFA Rules not processed for user `65d78df9d1e947f9a6cffd2f9e4be22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:57.418951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-78a9b919-123f-44d9-8171-26695d000724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:57.425654 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40377ac-e1ea-4d46-83f2-8fa8e54e75d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:57.426777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40377ac-e1ea-4d46-83f2-8fa8e54e75d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.427282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40377ac-e1ea-4d46-83f2-8fa8e54e75d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.427758 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40377ac-e1ea-4d46-83f2-8fa8e54e75d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.428636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40377ac-e1ea-4d46-83f2-8fa8e54e75d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:57.431223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40377ac-e1ea-4d46-83f2-8fa8e54e75d8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:57.484294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 660/1289] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:57 2022] POST /identity/v3/projects => generated 405 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:57.485565 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40377ac-e1ea-4d46-83f2-8fa8e54e75d8 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:57.495871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 630/1290] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:57.497324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe9cdb-88b5-425d-9f37-221abce0295d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.497606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe9cdb-88b5-425d-9f37-221abce0295d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.497878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe9cdb-88b5-425d-9f37-221abce0295d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.558687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-b9fe9cdb-88b5-425d-9f37-221abce0295d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:57.560849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b9fe9cdb-88b5-425d-9f37-221abce0295d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:57.569148 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40377ac-e1ea-4d46-83f2-8fa8e54e75d8 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:57.580991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916690da-7f68-4738-b08b-d6e95b00d1af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:57.580991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916690da-7f68-4738-b08b-d6e95b00d1af service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.580991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916690da-7f68-4738-b08b-d6e95b00d1af service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.580991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916690da-7f68-4738-b08b-d6e95b00d1af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.594054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:57.603604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:57.662525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 631/1291] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:57.664098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe9cdb-88b5-425d-9f37-221abce0295d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:57.668159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916690da-7f68-4738-b08b-d6e95b00d1af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:57.669671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916690da-7f68-4738-b08b-d6e95b00d1af service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:57.677548 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe9cdb-88b5-425d-9f37-221abce0295d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:57.684140 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca072518-3e45-4095-b505-d702657ba11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:57.684705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 661/1292] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:54:57.686295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ed1b4-7bc1-4a29-a5b0-02a892cb51c0 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.686630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ed1b4-7bc1-4a29-a5b0-02a892cb51c0 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.686908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ed1b4-7bc1-4a29-a5b0-02a892cb51c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.687126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca072518-3e45-4095-b505-d702657ba11b admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.687553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca072518-3e45-4095-b505-d702657ba11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.688079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca072518-3e45-4095-b505-d702657ba11b admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.689123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca072518-3e45-4095-b505-d702657ba11b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:57.691720 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca072518-3e45-4095-b505-d702657ba11b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:57.726081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-827ed1b4-7bc1-4a29-a5b0-02a892cb51c0 None None] MFA Rules not processed for user `bf96e2a560ae495e89da872a4f676c78`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:57.727714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-827ed1b4-7bc1-4a29-a5b0-02a892cb51c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:57.768919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 632/1293] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:57 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:57.768919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca072518-3e45-4095-b505-d702657ba11b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:57.860536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 662/1294] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:57.862045 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ed1b4-7bc1-4a29-a5b0-02a892cb51c0 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:57.879458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ed1b4-7bc1-4a29-a5b0-02a892cb51c0 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:57.880808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca072518-3e45-4095-b505-d702657ba11b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:57.889521 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4981e2-1fb1-40c1-b4e0-a0d2446e36a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:57.890224 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4981e2-1fb1-40c1-b4e0-a0d2446e36a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:57.890633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4981e2-1fb1-40c1-b4e0-a0d2446e36a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:57.891104 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4981e2-1fb1-40c1-b4e0-a0d2446e36a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:57.997602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4981e2-1fb1-40c1-b4e0-a0d2446e36a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:57.999677 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4981e2-1fb1-40c1-b4e0-a0d2446e36a5 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.012776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 633/1295] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:54:58.014715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890779ef-5824-48ad-bd18-5c83ae6a6d09 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.015180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890779ef-5824-48ad-bd18-5c83ae6a6d09 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.015809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890779ef-5824-48ad-bd18-5c83ae6a6d09 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.019265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 634/1296] 10.222.0.27 () {54 vars in 961 bytes} [Fri Oct 14 17:54:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:54:58.020154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890779ef-5824-48ad-bd18-5c83ae6a6d09 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.028406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a8bc3-ab10-42eb-9fbe-543516ac27cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.029050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a8bc3-ab10-42eb-9fbe-543516ac27cf admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.029325 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a8bc3-ab10-42eb-9fbe-543516ac27cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.029596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a8bc3-ab10-42eb-9fbe-543516ac27cf admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.030230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a8bc3-ab10-42eb-9fbe-543516ac27cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.032454 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a8bc3-ab10-42eb-9fbe-543516ac27cf admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.048527 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890779ef-5824-48ad-bd18-5c83ae6a6d09 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.062253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c577694-7164-477a-98e6-4765b4af5bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.063211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c577694-7164-477a-98e6-4765b4af5bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.064189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c577694-7164-477a-98e6-4765b4af5bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.064889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c577694-7164-477a-98e6-4765b4af5bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.067055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 663/1297] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:57 2022] POST /identity/v3/users => generated 309 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:58.068078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a8bc3-ab10-42eb-9fbe-543516ac27cf admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.085044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a8bc3-ab10-42eb-9fbe-543516ac27cf admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.092006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c577694-7164-477a-98e6-4765b4af5bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.093547 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c577694-7164-477a-98e6-4765b4af5bc0 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.102595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6d971-7403-4bc3-bbdc-c7ad3c73688f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.103730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6d971-7403-4bc3-bbdc-c7ad3c73688f admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.104046 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6d971-7403-4bc3-bbdc-c7ad3c73688f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.104310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6d971-7403-4bc3-bbdc-c7ad3c73688f admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.104943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6d971-7403-4bc3-bbdc-c7ad3c73688f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.110923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6d971-7403-4bc3-bbdc-c7ad3c73688f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.113519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 635/1298] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:54:58.114336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c577694-7164-477a-98e6-4765b4af5bc0 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.125340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c577694-7164-477a-98e6-4765b4af5bc0 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.132002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a13c54-d86a-4519-b0ce-feab7efa88b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.134912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a13c54-d86a-4519-b0ce-feab7efa88b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.135331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a13c54-d86a-4519-b0ce-feab7efa88b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.135604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a13c54-d86a-4519-b0ce-feab7efa88b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.136524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a13c54-d86a-4519-b0ce-feab7efa88b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.140127 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a13c54-d86a-4519-b0ce-feab7efa88b7 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.165006 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 664/1299] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:58 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:58.166321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93f289-165c-483d-9ab7-b8bf6aaae269 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.166602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93f289-165c-483d-9ab7-b8bf6aaae269 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.166930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93f289-165c-483d-9ab7-b8bf6aaae269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.191409 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 636/1300] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:54:58 2022] POST /identity/v3/projects => generated 403 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:58.194752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a13c54-d86a-4519-b0ce-feab7efa88b7 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.221106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:58.249452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:58.270987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2f93f289-165c-483d-9ab7-b8bf6aaae269 None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:58.274844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2f93f289-165c-483d-9ab7-b8bf6aaae269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:58.309224 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a13c54-d86a-4519-b0ce-feab7efa88b7 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.317984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c0ce8-5ee8-4f42-ba98-13f44ace9a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.319967 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c0ce8-5ee8-4f42-ba98-13f44ace9a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.320705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c0ce8-5ee8-4f42-ba98-13f44ace9a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.321264 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c0ce8-5ee8-4f42-ba98-13f44ace9a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.322347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c0ce8-5ee8-4f42-ba98-13f44ace9a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.324522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c0ce8-5ee8-4f42-ba98-13f44ace9a00 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.337847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 637/1301] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:58.338910 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c0ce8-5ee8-4f42-ba98-13f44ace9a00 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.355654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c0ce8-5ee8-4f42-ba98-13f44ace9a00 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.363644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959d2ff-7316-4e22-9f60-9b5bf368c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.365526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959d2ff-7316-4e22-9f60-9b5bf368c20f admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.365873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959d2ff-7316-4e22-9f60-9b5bf368c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.366266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959d2ff-7316-4e22-9f60-9b5bf368c20f admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.368676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959d2ff-7316-4e22-9f60-9b5bf368c20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.370651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959d2ff-7316-4e22-9f60-9b5bf368c20f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.399517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 638/1302] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:58.400635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959d2ff-7316-4e22-9f60-9b5bf368c20f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.425823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959d2ff-7316-4e22-9f60-9b5bf368c20f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.441615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bbe5c-f00a-43fe-9804-4821da525239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.442387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bbe5c-f00a-43fe-9804-4821da525239 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.442684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bbe5c-f00a-43fe-9804-4821da525239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.442974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bbe5c-f00a-43fe-9804-4821da525239 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.443585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bbe5c-f00a-43fe-9804-4821da525239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.446123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bbe5c-f00a-43fe-9804-4821da525239 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.450497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 665/1303] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Oct 14 17:54:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:58.452589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93f289-165c-483d-9ab7-b8bf6aaae269 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.484924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93f289-165c-483d-9ab7-b8bf6aaae269 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.497737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 639/1304] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:54:58 2022] POST /identity/v3/users => generated 308 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:54:58.497737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bbe5c-f00a-43fe-9804-4821da525239 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.501284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8be4bf-2153-47dc-a153-ff980524d5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.501643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8be4bf-2153-47dc-a153-ff980524d5b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.501904 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8be4bf-2153-47dc-a153-ff980524d5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.502180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8be4bf-2153-47dc-a153-ff980524d5b8 admin admin] PATH_INFO: `/v3/projects/6e85572599034fda80eecccd010d408d/users/c59fcf67679648bfb0e7db15b517ab89/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.597282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bbe5c-f00a-43fe-9804-4821da525239 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.597282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e4e09-cc55-4a14-b890-ec3cee277675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.597282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e4e09-cc55-4a14-b890-ec3cee277675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.597282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e4e09-cc55-4a14-b890-ec3cee277675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.601665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e4e09-cc55-4a14-b890-ec3cee277675 admin admin] PATH_INFO: `/v3/projects/9b4cf02e97f9472cb5075dee9a1baa14/users/19a2aad4a4d843ddbeba43dd8cadc93d/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.602192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8be4bf-2153-47dc-a153-ff980524d5b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e85572599034fda80eecccd010d408d, user_id=c59fcf67679648bfb0e7db15b517ab89, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.602192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8be4bf-2153-47dc-a153-ff980524d5b8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.602992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e4e09-cc55-4a14-b890-ec3cee277675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4cf02e97f9472cb5075dee9a1baa14, user_id=19a2aad4a4d843ddbeba43dd8cadc93d, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.605464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e4e09-cc55-4a14-b890-ec3cee277675 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.630579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 666/1306] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:58 2022] PUT /identity/v3/projects/6e85572599034fda80eecccd010d408d/users/c59fcf67679648bfb0e7db15b517ab89/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:58.631751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8be4bf-2153-47dc-a153-ff980524d5b8 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.634932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 640/1306] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:58 2022] PUT /identity/v3/projects/9b4cf02e97f9472cb5075dee9a1baa14/users/19a2aad4a4d843ddbeba43dd8cadc93d/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:58.636405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e4e09-cc55-4a14-b890-ec3cee277675 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.747143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e4e09-cc55-4a14-b890-ec3cee277675 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.761762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9a8c4-eb01-4cd5-a2ec-1750c0707c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.762066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a8c4-eb01-4cd5-a2ec-1750c0707c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.762295 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a8c4-eb01-4cd5-a2ec-1750c0707c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.763744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a8c4-eb01-4cd5-a2ec-1750c0707c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.764859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9a8c4-eb01-4cd5-a2ec-1750c0707c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.769241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9a8c4-eb01-4cd5-a2ec-1750c0707c4e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.795565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 641/1307] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:58.796387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9a8c4-eb01-4cd5-a2ec-1750c0707c4e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.801638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8be4bf-2153-47dc-a153-ff980524d5b8 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.818178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9a8c4-eb01-4cd5-a2ec-1750c0707c4e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.838296 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb96031-2bfa-46fe-be55-393a210373c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.838925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb96031-2bfa-46fe-be55-393a210373c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.839204 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb96031-2bfa-46fe-be55-393a210373c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.839481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb96031-2bfa-46fe-be55-393a210373c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.840156 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb96031-2bfa-46fe-be55-393a210373c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.841763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb96031-2bfa-46fe-be55-393a210373c3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.857333 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 642/1308] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:58.857333 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb96031-2bfa-46fe-be55-393a210373c3 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.871787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb96031-2bfa-46fe-be55-393a210373c3 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.877641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464d57e-0e66-4c95-9b3e-6bcb1acd8a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.878640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464d57e-0e66-4c95-9b3e-6bcb1acd8a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.878841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464d57e-0e66-4c95-9b3e-6bcb1acd8a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.879101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464d57e-0e66-4c95-9b3e-6bcb1acd8a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.880127 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464d57e-0e66-4c95-9b3e-6bcb1acd8a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.882002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464d57e-0e66-4c95-9b3e-6bcb1acd8a13 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.896833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 643/1309] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:58.897895 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464d57e-0e66-4c95-9b3e-6bcb1acd8a13 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.902638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba87108-6d79-4c96-a913-21390ce0d655 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.904626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba87108-6d79-4c96-a913-21390ce0d655 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.905017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba87108-6d79-4c96-a913-21390ce0d655 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.906201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba87108-6d79-4c96-a913-21390ce0d655 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.913209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464d57e-0e66-4c95-9b3e-6bcb1acd8a13 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:58.914922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:58.922184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b69501-b437-40ce-9df9-1c904c119566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:58.923871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b69501-b437-40ce-9df9-1c904c119566 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:58.924416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b69501-b437-40ce-9df9-1c904c119566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:58.925029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b69501-b437-40ce-9df9-1c904c119566 admin admin] PATH_INFO: `/v3/projects/50048b7509fe4bc89b4cff10cacc4681/users/3597d46e97d6497fa758bb0dab9d5743/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:58.944726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b69501-b437-40ce-9df9-1c904c119566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50048b7509fe4bc89b4cff10cacc4681, user_id=3597d46e97d6497fa758bb0dab9d5743, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:58.948777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b69501-b437-40ce-9df9-1c904c119566 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:58.972259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 644/1310] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:58 2022] PUT /identity/v3/projects/50048b7509fe4bc89b4cff10cacc4681/users/3597d46e97d6497fa758bb0dab9d5743/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:58.973380 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b69501-b437-40ce-9df9-1c904c119566 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:58.988809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:59.020216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba87108-6d79-4c96-a913-21390ce0d655 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:59.023811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba87108-6d79-4c96-a913-21390ce0d655 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:59.043583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 667/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:54:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:54:59.074120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba87108-6d79-4c96-a913-21390ce0d655 service nova] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:59.075533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b69501-b437-40ce-9df9-1c904c119566 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:59.084392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880aa5d-6b04-4ea5-a334-db76bdf25b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:59.085156 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880aa5d-6b04-4ea5-a334-db76bdf25b4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:59.085497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880aa5d-6b04-4ea5-a334-db76bdf25b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:59.085789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880aa5d-6b04-4ea5-a334-db76bdf25b4f admin admin] PATH_INFO: `/v3/projects/9b4cf02e97f9472cb5075dee9a1baa14/users/19a2aad4a4d843ddbeba43dd8cadc93d/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:59.089575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880aa5d-6b04-4ea5-a334-db76bdf25b4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4cf02e97f9472cb5075dee9a1baa14, user_id=19a2aad4a4d843ddbeba43dd8cadc93d, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:59.093749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880aa5d-6b04-4ea5-a334-db76bdf25b4f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:59.106961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba87108-6d79-4c96-a913-21390ce0d655 service nova] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:59.129122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 645/1312] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:58 2022] PUT /identity/v3/projects/9b4cf02e97f9472cb5075dee9a1baa14/users/19a2aad4a4d843ddbeba43dd8cadc93d/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:59.130147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880aa5d-6b04-4ea5-a334-db76bdf25b4f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:59.262432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fd767-06ba-4536-aaff-15e7f6642e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:59.263451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fd767-06ba-4536-aaff-15e7f6642e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:59.264306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fd767-06ba-4536-aaff-15e7f6642e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:59.264306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fd767-06ba-4536-aaff-15e7f6642e29 admin admin] PATH_INFO: `/v3/projects/6e85572599034fda80eecccd010d408d/users/c59fcf67679648bfb0e7db15b517ab89/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:59.268467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fd767-06ba-4536-aaff-15e7f6642e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e85572599034fda80eecccd010d408d, user_id=c59fcf67679648bfb0e7db15b517ab89, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:59.276025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fd767-06ba-4536-aaff-15e7f6642e29 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:59.314051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 668/1313] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:59 2022] PUT /identity/v3/projects/6e85572599034fda80eecccd010d408d/users/c59fcf67679648bfb0e7db15b517ab89/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:59.314051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d18c42-fb73-4f34-9d12-abc715b4a7e7 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:59.314051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d18c42-fb73-4f34-9d12-abc715b4a7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:59.314051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d18c42-fb73-4f34-9d12-abc715b4a7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:59.326991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880aa5d-6b04-4ea5-a334-db76bdf25b4f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:59.417751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c6d18c42-fb73-4f34-9d12-abc715b4a7e7 None None] MFA Rules not processed for user `19a2aad4a4d843ddbeba43dd8cadc93d`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:59.424697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c6d18c42-fb73-4f34-9d12-abc715b4a7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:59.489065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96e199-0218-4715-b639-cdc73a4daa31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:59.489839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96e199-0218-4715-b639-cdc73a4daa31 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:59.498778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96e199-0218-4715-b639-cdc73a4daa31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:59.498778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96e199-0218-4715-b639-cdc73a4daa31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:59.501222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96e199-0218-4715-b639-cdc73a4daa31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:59.504715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96e199-0218-4715-b639-cdc73a4daa31 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:59.543788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 646/1314] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:54:59.545358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461a3f2-9525-4dc8-ba93-38c98aeed343 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:59.545645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461a3f2-9525-4dc8-ba93-38c98aeed343 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:59.545916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461a3f2-9525-4dc8-ba93-38c98aeed343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:59.620511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 669/1315] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:59.624467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d18c42-fb73-4f34-9d12-abc715b4a7e7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:59.624598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a461a3f2-9525-4dc8-ba93-38c98aeed343 None None] MFA Rules not processed for user `c59fcf67679648bfb0e7db15b517ab89`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:59.624598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a461a3f2-9525-4dc8-ba93-38c98aeed343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:59.659392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d18c42-fb73-4f34-9d12-abc715b4a7e7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:59.682519 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1457833-abc2-45b4-b96d-bc53a14e608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:59.683807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1457833-abc2-45b4-b96d-bc53a14e608e admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:59.684112 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1457833-abc2-45b4-b96d-bc53a14e608e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:59.684410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1457833-abc2-45b4-b96d-bc53a14e608e admin admin] PATH_INFO: `/v3/projects/50048b7509fe4bc89b4cff10cacc4681/users/3597d46e97d6497fa758bb0dab9d5743/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:59.686696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1457833-abc2-45b4-b96d-bc53a14e608e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50048b7509fe4bc89b4cff10cacc4681, user_id=3597d46e97d6497fa758bb0dab9d5743, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:59.690115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1457833-abc2-45b4-b96d-bc53a14e608e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:59.711740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 670/1316] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:54:59 2022] PUT /identity/v3/projects/50048b7509fe4bc89b4cff10cacc4681/users/3597d46e97d6497fa758bb0dab9d5743/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:54:59.713097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1457833-abc2-45b4-b96d-bc53a14e608e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:59.722800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:59.733123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:54:59.789092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 647/1317] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:54:59 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:54:59.790939 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ffc6-77f8-4cab-8f33-5ccadc2a7f29 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:59.791382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ffc6-77f8-4cab-8f33-5ccadc2a7f29 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:59.792187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ffc6-77f8-4cab-8f33-5ccadc2a7f29 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:59.794733 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 648/1318] 10.222.0.27 () {54 vars in 912 bytes} [Fri Oct 14 17:54:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:54:59.802241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955cfb-a1ab-4ef4-bb39-45a89b16898b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:59.803300 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955cfb-a1ab-4ef4-bb39-45a89b16898b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:59.803790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955cfb-a1ab-4ef4-bb39-45a89b16898b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:59.822971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1457833-abc2-45b4-b96d-bc53a14e608e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:59.828616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875a689-369c-4cce-9ef4-bf9e3dc2c864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:59.829132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a689-369c-4cce-9ef4-bf9e3dc2c864 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:59.829558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a689-369c-4cce-9ef4-bf9e3dc2c864 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:59.829762 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a689-369c-4cce-9ef4-bf9e3dc2c864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:59.834322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a2955cfb-a1ab-4ef4-bb39-45a89b16898b None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:54:59.836687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a2955cfb-a1ab-4ef4-bb39-45a89b16898b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:54:59.922092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875a689-369c-4cce-9ef4-bf9e3dc2c864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:54:59.925080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875a689-369c-4cce-9ef4-bf9e3dc2c864 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:54:59.940470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 671/1319] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:54:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:54:59.942343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875a689-369c-4cce-9ef4-bf9e3dc2c864 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:54:59.953961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875a689-369c-4cce-9ef4-bf9e3dc2c864 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:54:59.960502 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a88f2-b10a-4365-8bd4-cc6a1e05e6a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:54:59.962706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a88f2-b10a-4365-8bd4-cc6a1e05e6a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:54:59.964114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a88f2-b10a-4365-8bd4-cc6a1e05e6a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:54:59.964742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a88f2-b10a-4365-8bd4-cc6a1e05e6a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:54:59.998832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 649/1320] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:54:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:00.141918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a88f2-b10a-4365-8bd4-cc6a1e05e6a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:00.144114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a88f2-b10a-4365-8bd4-cc6a1e05e6a6 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:00.158506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 672/1321] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:54:59 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:00.296135 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2955cfb-a1ab-4ef4-bb39-45a89b16898b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:00.317956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2955cfb-a1ab-4ef4-bb39-45a89b16898b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:00.325498 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d045a2-8e5c-436e-86b6-93bd52787916 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:00.326140 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d045a2-8e5c-436e-86b6-93bd52787916 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:00.326436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d045a2-8e5c-436e-86b6-93bd52787916 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:00.328747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d045a2-8e5c-436e-86b6-93bd52787916 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:00.360215 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d045a2-8e5c-436e-86b6-93bd52787916 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:00.361847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d045a2-8e5c-436e-86b6-93bd52787916 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:00.400783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 650/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:01.305867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a88f2-b10a-4365-8bd4-cc6a1e05e6a6 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:01.478634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a88f2-b10a-4365-8bd4-cc6a1e05e6a6 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:01.496595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:01.497628 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:01.498139 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:01.498650 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] PATH_INFO: `/v3/users/8dddbcca59ae44ff9aeb7e274d1e2558` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:01.500847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dddbcca59ae44ff9aeb7e274d1e2558)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:01.503234 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:01.510116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:01.622998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dddbcca59ae44ff9aeb7e274d1e2558'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:01.624364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dddbcca59ae44ff9aeb7e274d1e2558'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:01.640221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dddbcca59ae44ff9aeb7e274d1e2558'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:01.641035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dddbcca59ae44ff9aeb7e274d1e2558'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:01.657350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1645fdc2-755a-4a93-87f9-e35e021557dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dddbcca59ae44ff9aeb7e274d1e2558'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:01.678275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 673/1323] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:01 2022] DELETE /identity/v3/users/8dddbcca59ae44ff9aeb7e274d1e2558 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:01.680186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180724-a458-4379-8078-35505a0da1ed None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:01.680843 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180724-a458-4379-8078-35505a0da1ed None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:01.680843 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180724-a458-4379-8078-35505a0da1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:01.721959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-b4180724-a458-4379-8078-35505a0da1ed None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:01.723601 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b4180724-a458-4379-8078-35505a0da1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:01.844487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 651/1324] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:01.914159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9d082-5b8f-4904-ad3a-5a80ac03eb33 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:01.914699 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9d082-5b8f-4904-ad3a-5a80ac03eb33 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:01.915282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9d082-5b8f-4904-ad3a-5a80ac03eb33 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:01.917232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 674/1325] 10.222.0.27 () {54 vars in 961 bytes} [Fri Oct 14 17:55:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:55:01.939708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3360d0-51f8-455c-a291-4b16da147682 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:01.940312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3360d0-51f8-455c-a291-4b16da147682 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:01.940776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3360d0-51f8-455c-a291-4b16da147682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:01.966674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9d082-5b8f-4904-ad3a-5a80ac03eb33 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:01.969162 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-40e9d082-5b8f-4904-ad3a-5a80ac03eb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:02.014975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1e3360d0-51f8-455c-a291-4b16da147682 None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:02.017626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1e3360d0-51f8-455c-a291-4b16da147682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:02.140458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9d082-5b8f-4904-ad3a-5a80ac03eb33 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:02.149811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:02.151285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70054f23-eda6-4c86-b265-d432683ed314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:02.151891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70054f23-eda6-4c86-b265-d432683ed314 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:02.154816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70054f23-eda6-4c86-b265-d432683ed314 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:02.155345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70054f23-eda6-4c86-b265-d432683ed314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:02.180284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70054f23-eda6-4c86-b265-d432683ed314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:02.181586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70054f23-eda6-4c86-b265-d432683ed314 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:02.216154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 675/1326] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:02.244752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 652/1327] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Oct 14 17:55:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:02.250972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec37bc1-e095-4e93-b18f-951783252017 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:02.251295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec37bc1-e095-4e93-b18f-951783252017 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:02.251626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec37bc1-e095-4e93-b18f-951783252017 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:02.253706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 676/1328] 10.222.0.27 () {54 vars in 912 bytes} [Fri Oct 14 17:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:55:02.260722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367f57c-1d01-480c-83ca-cacd5ec02f16 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:02.261143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367f57c-1d01-480c-83ca-cacd5ec02f16 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:02.262376 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367f57c-1d01-480c-83ca-cacd5ec02f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:02.303518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2367f57c-1d01-480c-83ca-cacd5ec02f16 None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:02.305710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2367f57c-1d01-480c-83ca-cacd5ec02f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:02.356294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 653/1329] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:55:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:02.376372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec37bc1-e095-4e93-b18f-951783252017 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:02.379223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4ec37bc1-e095-4e93-b18f-951783252017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:02.532676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec37bc1-e095-4e93-b18f-951783252017 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:02.537630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a13d5-f055-4898-a0a8-918ad95ecb7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:02.538251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a13d5-f055-4898-a0a8-918ad95ecb7d service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:02.538567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a13d5-f055-4898-a0a8-918ad95ecb7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:02.538847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a13d5-f055-4898-a0a8-918ad95ecb7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:02.564337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a13d5-f055-4898-a0a8-918ad95ecb7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:02.567918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a13d5-f055-4898-a0a8-918ad95ecb7d service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:02.613859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 677/1330] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:03.817003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5677ea-ab1c-42e3-9e58-32d9fa747bb0 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:03.817598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5677ea-ab1c-42e3-9e58-32d9fa747bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:03.817598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5677ea-ab1c-42e3-9e58-32d9fa747bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:03.864262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-cd5677ea-ab1c-42e3-9e58-32d9fa747bb0 None None] MFA Rules not processed for user `3597d46e97d6497fa758bb0dab9d5743`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:03.865501 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-cd5677ea-ab1c-42e3-9e58-32d9fa747bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:03.990786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a13d5-f055-4898-a0a8-918ad95ecb7d service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:03.992427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-263a13d5-f055-4898-a0a8-918ad95ecb7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:04.004716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 654/1331] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:03 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:04.029397 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a13d5-f055-4898-a0a8-918ad95ecb7d service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:04.053925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:04.054577 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:04.054914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:04.055376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] PATH_INFO: `/v3/projects/a0460ab6e3a54a7a9d95da330b625879` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:04.057993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0460ab6e3a54a7a9d95da330b625879)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:04.059244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: WARNING py.warnings [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:55:04.059244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: warnings.warn(msg) Oct 14 17:55:04.059244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: Oct 14 17:55:04.061137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:04.120781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:04.186282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0460ab6e3a54a7a9d95da330b625879', 'request_id': 'req-e626e242-2f22-4770-9931-82d95388f88b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.211453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0460ab6e3a54a7a9d95da330b625879', 'request_id': 'req-e626e242-2f22-4770-9931-82d95388f88b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.215254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5677ea-ab1c-42e3-9e58-32d9fa747bb0 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:04.216532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-cd5677ea-ab1c-42e3-9e58-32d9fa747bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:04.238411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0460ab6e3a54a7a9d95da330b625879', 'request_id': 'req-e626e242-2f22-4770-9931-82d95388f88b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.250988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5677ea-ab1c-42e3-9e58-32d9fa747bb0 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:04.278518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] The token cache is being invalidate because project a0460ab6e3a54a7a9d95da330b625879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:04.278915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.284183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.286485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 678/1332] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:03 2022] DELETE /identity/v3/projects/a0460ab6e3a54a7a9d95da330b625879 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:04.290901 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:04.292761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:04.450047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:04.450737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:04.451018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:04.456972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] PATH_INFO: `/v3/users/65d78df9d1e947f9a6cffd2f9e4be22e` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:04.458342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d78df9d1e947f9a6cffd2f9e4be22e)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:04.462180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: WARNING py.warnings [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 17:55:04.462180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: warnings.warn(msg) Oct 14 17:55:04.462180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: Oct 14 17:55:04.465502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:04.477829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626e242-2f22-4770-9931-82d95388f88b admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:04.500075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:04.500881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:04.501204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:04.501471 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] PATH_INFO: `/v3/users/c59fcf67679648bfb0e7db15b517ab89` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:04.505064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c59fcf67679648bfb0e7db15b517ab89)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:04.511377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:04.678606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c59fcf67679648bfb0e7db15b517ab89'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.681014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c59fcf67679648bfb0e7db15b517ab89'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.688011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d78df9d1e947f9a6cffd2f9e4be22e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.689369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d78df9d1e947f9a6cffd2f9e4be22e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.706263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c59fcf67679648bfb0e7db15b517ab89'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.706263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c59fcf67679648bfb0e7db15b517ab89'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.722779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d78df9d1e947f9a6cffd2f9e4be22e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.724282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d78df9d1e947f9a6cffd2f9e4be22e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.735866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1be91b2f-8407-4f2e-96d2-bb79a38f9950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c59fcf67679648bfb0e7db15b517ab89'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.751627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:04.785635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 679/1333] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:04 2022] DELETE /identity/v3/users/c59fcf67679648bfb0e7db15b517ab89 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:04.803155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d78df9d1e947f9a6cffd2f9e4be22e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:04.848576 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 655/1334] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:04 2022] DELETE /identity/v3/users/65d78df9d1e947f9a6cffd2f9e4be22e => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:04.850565 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b99c1-369a-47f7-b4e0-f09ee6b54b52 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:04.850860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b99c1-369a-47f7-b4e0-f09ee6b54b52 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:04.851137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b99c1-369a-47f7-b4e0-f09ee6b54b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:04.905438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-d83b99c1-369a-47f7-b4e0-f09ee6b54b52 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:04.909022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d83b99c1-369a-47f7-b4e0-f09ee6b54b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:05.048042 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:05.211432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 680/1335] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:05.356059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:05.358432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:05.521788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd52c71-526f-41bd-a64e-1d4d35de31ff admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:05.529967 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba8bc7-611c-4e04-a8a7-2971894fcf6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:05.531101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba8bc7-611c-4e04-a8a7-2971894fcf6d service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:05.531101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba8bc7-611c-4e04-a8a7-2971894fcf6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:05.532408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba8bc7-611c-4e04-a8a7-2971894fcf6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:05.554767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba8bc7-611c-4e04-a8a7-2971894fcf6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:05.557081 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba8bc7-611c-4e04-a8a7-2971894fcf6d service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:05.610693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 656/1336] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:06.385918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b99c1-369a-47f7-b4e0-f09ee6b54b52 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:06.387528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d83b99c1-369a-47f7-b4e0-f09ee6b54b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:06.411393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b99c1-369a-47f7-b4e0-f09ee6b54b52 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:06.420465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:06.421149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:06.421374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:06.421648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] PATH_INFO: `/v3/projects/6e85572599034fda80eecccd010d408d` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:06.422917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e85572599034fda80eecccd010d408d)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:06.425398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:06.525787 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e85572599034fda80eecccd010d408d', 'request_id': 'req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:06.532914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e85572599034fda80eecccd010d408d', 'request_id': 'req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:06.543922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e85572599034fda80eecccd010d408d', 'request_id': 'req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:06.566772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] The token cache is being invalidate because project 6e85572599034fda80eecccd010d408d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:06.567312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:06.568350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-db1e5f8e-3452-4bd9-a9a0-bf3117fc2829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:06.571220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 681/1337] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:06 2022] DELETE /identity/v3/projects/6e85572599034fda80eecccd010d408d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:06.575845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4036825-93c8-438f-b716-661bc7157a38 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:06.579207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4036825-93c8-438f-b716-661bc7157a38 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:06.579808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4036825-93c8-438f-b716-661bc7157a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:06.612068 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c4036825-93c8-438f-b716-661bc7157a38 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:06.614122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c4036825-93c8-438f-b716-661bc7157a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:06.730473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 657/1338] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:06.735411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4c14-5793-4ae6-be5e-971b63a5b69e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:06.735695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4c14-5793-4ae6-be5e-971b63a5b69e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:06.736007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4c14-5793-4ae6-be5e-971b63a5b69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:06.784096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ef9e4c14-5793-4ae6-be5e-971b63a5b69e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:06.787298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ef9e4c14-5793-4ae6-be5e-971b63a5b69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:06.802245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 682/1339] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:06.807322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4036825-93c8-438f-b716-661bc7157a38 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:06.823640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4036825-93c8-438f-b716-661bc7157a38 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:06.832686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb35bdc-cfe2-4c67-b07b-2ca5d56db5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:06.833619 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb35bdc-cfe2-4c67-b07b-2ca5d56db5cd admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:06.835001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb35bdc-cfe2-4c67-b07b-2ca5d56db5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:06.835464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb35bdc-cfe2-4c67-b07b-2ca5d56db5cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:06.837005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb35bdc-cfe2-4c67-b07b-2ca5d56db5cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:06.837630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb35bdc-cfe2-4c67-b07b-2ca5d56db5cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:06.839956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb35bdc-cfe2-4c67-b07b-2ca5d56db5cd admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:06.857456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 658/1340] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:06.862209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e4c14-5793-4ae6-be5e-971b63a5b69e None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:06.878156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e4c14-5793-4ae6-be5e-971b63a5b69e None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:06.892419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a355c-7cad-4df8-acff-24ffe6c4a2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:06.893053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a355c-7cad-4df8-acff-24ffe6c4a2b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:06.893372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a355c-7cad-4df8-acff-24ffe6c4a2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:06.893595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a355c-7cad-4df8-acff-24ffe6c4a2b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:06.894256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a355c-7cad-4df8-acff-24ffe6c4a2b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:06.896189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a355c-7cad-4df8-acff-24ffe6c4a2b0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:06.910690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12535a39-a9d4-485d-9c5c-ff2a02e93e50 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:06.911042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12535a39-a9d4-485d-9c5c-ff2a02e93e50 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:06.911244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12535a39-a9d4-485d-9c5c-ff2a02e93e50 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:06.912760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 659/1341] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:55:06.965843 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12535a39-a9d4-485d-9c5c-ff2a02e93e50 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:06.975110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 683/1342] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:06 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:07.038291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eff01-8adf-4350-ab0b-db8338526a8c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:07.039073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eff01-8adf-4350-ab0b-db8338526a8c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:07.039602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eff01-8adf-4350-ab0b-db8338526a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:07.140449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-d43eff01-8adf-4350-ab0b-db8338526a8c None None] MFA Rules not processed for user `73089bc9f85a405b86c4cd3f8cfa830a`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:07.144983 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d43eff01-8adf-4350-ab0b-db8338526a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:07.146637 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12535a39-a9d4-485d-9c5c-ff2a02e93e50 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:07.158273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef5566-aa5c-4f53-82c3-97a240589662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:07.158901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef5566-aa5c-4f53-82c3-97a240589662 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:07.159202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef5566-aa5c-4f53-82c3-97a240589662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:07.159476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef5566-aa5c-4f53-82c3-97a240589662 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:07.160199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef5566-aa5c-4f53-82c3-97a240589662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:07.162418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef5566-aa5c-4f53-82c3-97a240589662 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:07.211609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:07.230858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 660/1343] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:06 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:07.233866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef5566-aa5c-4f53-82c3-97a240589662 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:07.265410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef5566-aa5c-4f53-82c3-97a240589662 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:07.281208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955260c4-2f68-417b-aaf5-d88217657cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:07.283965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955260c4-2f68-417b-aaf5-d88217657cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:07.284666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955260c4-2f68-417b-aaf5-d88217657cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:07.285924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955260c4-2f68-417b-aaf5-d88217657cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:07.287313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955260c4-2f68-417b-aaf5-d88217657cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:07.289592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955260c4-2f68-417b-aaf5-d88217657cb9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:07.319686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 661/1344] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:07.324808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955260c4-2f68-417b-aaf5-d88217657cb9 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:07.339733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 684/1345] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:55:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:07.348371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955260c4-2f68-417b-aaf5-d88217657cb9 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:07.351103 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5b1c2-2470-40a5-8cd9-170219c288cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:07.352343 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5b1c2-2470-40a5-8cd9-170219c288cb admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:07.352897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5b1c2-2470-40a5-8cd9-170219c288cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:07.353279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5b1c2-2470-40a5-8cd9-170219c288cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:07.355032 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5b1c2-2470-40a5-8cd9-170219c288cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:07.357453 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5b1c2-2470-40a5-8cd9-170219c288cb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:07.377238 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eff01-8adf-4350-ab0b-db8338526a8c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:07.389784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 662/1346] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:07.392646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eff01-8adf-4350-ab0b-db8338526a8c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:07.401186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b0af1-1c7a-4866-b624-a8cbba81104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:07.402334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b0af1-1c7a-4866-b624-a8cbba81104c admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:07.402936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b0af1-1c7a-4866-b624-a8cbba81104c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:07.403534 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b0af1-1c7a-4866-b624-a8cbba81104c admin admin] PATH_INFO: `/v3/projects/28c0ea32e702484593f155ceb4d4b3c0/users/e9b6594a78e240fc9bbb50babd7a1c1a/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:07.430255 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b0af1-1c7a-4866-b624-a8cbba81104c admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c0ea32e702484593f155ceb4d4b3c0, user_id=e9b6594a78e240fc9bbb50babd7a1c1a, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:07.435603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b0af1-1c7a-4866-b624-a8cbba81104c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:07.440865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:07.448340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5b1c2-2470-40a5-8cd9-170219c288cb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:07.452720 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-65a5b1c2-2470-40a5-8cd9-170219c288cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:07.474790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5b1c2-2470-40a5-8cd9-170219c288cb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:07.478326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 685/1347] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:07 2022] PUT /identity/v3/projects/28c0ea32e702484593f155ceb4d4b3c0/users/e9b6594a78e240fc9bbb50babd7a1c1a/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:07.489467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0990fe-ab27-4b63-9b17-32e3dc8d8901 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:07.489748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0990fe-ab27-4b63-9b17-32e3dc8d8901 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:07.492779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0990fe-ab27-4b63-9b17-32e3dc8d8901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:07.557094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-fb0990fe-ab27-4b63-9b17-32e3dc8d8901 None None] MFA Rules not processed for user `e9b6594a78e240fc9bbb50babd7a1c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:07.558365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-fb0990fe-ab27-4b63-9b17-32e3dc8d8901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:07.606260 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:07.608259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:07.608650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:07.610763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] PATH_INFO: `/v3/projects/7c55aced11be4d46890f8d417890c067` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:07.612578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c55aced11be4d46890f8d417890c067)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:07.616691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:07.781512 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c55aced11be4d46890f8d417890c067', 'request_id': 'req-7781732d-0467-4838-94de-7b60abef1417'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:07.783823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 686/1348] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:07.785273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0990fe-ab27-4b63-9b17-32e3dc8d8901 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:07.787953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-fb0990fe-ab27-4b63-9b17-32e3dc8d8901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:07.791643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c55aced11be4d46890f8d417890c067', 'request_id': 'req-7781732d-0467-4838-94de-7b60abef1417'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:07.812786 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c55aced11be4d46890f8d417890c067', 'request_id': 'req-7781732d-0467-4838-94de-7b60abef1417'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:07.835005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] The token cache is being invalidate because project 7c55aced11be4d46890f8d417890c067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:07.835336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:07.847166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:07.853323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 663/1349] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:07 2022] DELETE /identity/v3/projects/7c55aced11be4d46890f8d417890c067 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:07.854454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:07.856028 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:07.947470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0990fe-ab27-4b63-9b17-32e3dc8d8901 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:07.968985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781732d-0467-4838-94de-7b60abef1417 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:07.973663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bc079-8727-4dee-930d-f76142ce03d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:07.974920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bc079-8727-4dee-930d-f76142ce03d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:07.975276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bc079-8727-4dee-930d-f76142ce03d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:07.975614 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bc079-8727-4dee-930d-f76142ce03d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:07.978847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-648bc079-8727-4dee-930d-f76142ce03d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:08.049354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:08.126391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e248664-8e40-4f45-9899-056fbd45ee2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:08.127024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e248664-8e40-4f45-9899-056fbd45ee2e service placement] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:08.127300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e248664-8e40-4f45-9899-056fbd45ee2e service placement] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:08.127564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e248664-8e40-4f45-9899-056fbd45ee2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:08.132630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1e248664-8e40-4f45-9899-056fbd45ee2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:08.151894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bc079-8727-4dee-930d-f76142ce03d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:08.153358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bc079-8727-4dee-930d-f76142ce03d4 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:08.165188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e248664-8e40-4f45-9899-056fbd45ee2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:08.166799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e248664-8e40-4f45-9899-056fbd45ee2e service placement] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:08.181567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 664/1350] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:08.184503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bc079-8727-4dee-930d-f76142ce03d4 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:08.189789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-648bc079-8727-4dee-930d-f76142ce03d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:08.193076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 687/1351] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:08.194434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266171ca-3d69-4b96-aaa0-c58c49028bd4 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:08.194710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266171ca-3d69-4b96-aaa0-c58c49028bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:08.194980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266171ca-3d69-4b96-aaa0-c58c49028bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:08.245743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-266171ca-3d69-4b96-aaa0-c58c49028bd4 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:08.247252 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-266171ca-3d69-4b96-aaa0-c58c49028bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:08.274456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:08.364141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bc079-8727-4dee-930d-f76142ce03d4 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:08.376429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341668b-f0c6-41a7-94a4-e03055302250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:08.379640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341668b-f0c6-41a7-94a4-e03055302250 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:08.379847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341668b-f0c6-41a7-94a4-e03055302250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:08.381584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341668b-f0c6-41a7-94a4-e03055302250 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:08.382044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341668b-f0c6-41a7-94a4-e03055302250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:08.386324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341668b-f0c6-41a7-94a4-e03055302250 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:08.425838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 688/1352] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:08.432543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4fc16-0798-4f54-96e6-4dfdd9f37c82 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:08.432904 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4fc16-0798-4f54-96e6-4dfdd9f37c82 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:08.433243 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4fc16-0798-4f54-96e6-4dfdd9f37c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:08.472535 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 665/1353] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:08 2022] POST /identity/v3/projects => generated 385 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:08.478563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341668b-f0c6-41a7-94a4-e03055302250 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:08.481947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0ad4fc16-0798-4f54-96e6-4dfdd9f37c82 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:08.483685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0ad4fc16-0798-4f54-96e6-4dfdd9f37c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:08.650711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341668b-f0c6-41a7-94a4-e03055302250 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:08.652816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 689/1354] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:08.657386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4fc16-0798-4f54-96e6-4dfdd9f37c82 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:08.668837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdc877-b464-4342-b053-b6f91ac21198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:08.669484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdc877-b464-4342-b053-b6f91ac21198 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:08.669764 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdc877-b464-4342-b053-b6f91ac21198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:08.670037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdc877-b464-4342-b053-b6f91ac21198 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:08.674658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdc877-b464-4342-b053-b6f91ac21198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:08.678146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4fc16-0798-4f54-96e6-4dfdd9f37c82 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:08.683601 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdc877-b464-4342-b053-b6f91ac21198 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:08.694084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f8eb6-199f-44d6-8975-c9ddbe6a5eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:08.694710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f8eb6-199f-44d6-8975-c9ddbe6a5eba admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:08.694979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f8eb6-199f-44d6-8975-c9ddbe6a5eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:08.695796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f8eb6-199f-44d6-8975-c9ddbe6a5eba admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:08.696491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f8eb6-199f-44d6-8975-c9ddbe6a5eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:08.696785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f8eb6-199f-44d6-8975-c9ddbe6a5eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:08.698447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f8eb6-199f-44d6-8975-c9ddbe6a5eba admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:08.728815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 690/1355] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:08.733015 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f8eb6-199f-44d6-8975-c9ddbe6a5eba admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:08.761397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 666/1356] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:08 2022] POST /identity/v3/users => generated 299 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:55:08.769756 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdc877-b464-4342-b053-b6f91ac21198 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:08.771923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f8eb6-199f-44d6-8975-c9ddbe6a5eba admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:08.802029 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc0471-b81e-4d0e-aef0-06f3a4f75aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:08.802683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc0471-b81e-4d0e-aef0-06f3a4f75aaa admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:08.802951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc0471-b81e-4d0e-aef0-06f3a4f75aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:08.803217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc0471-b81e-4d0e-aef0-06f3a4f75aaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:08.803850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc0471-b81e-4d0e-aef0-06f3a4f75aaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:08.808965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdc877-b464-4342-b053-b6f91ac21198 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:08.810800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc0471-b81e-4d0e-aef0-06f3a4f75aaa admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:08.825417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1676399-d760-4201-a95c-b0ab9672bfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:08.828774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1676399-d760-4201-a95c-b0ab9672bfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:08.829113 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1676399-d760-4201-a95c-b0ab9672bfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:08.829502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1676399-d760-4201-a95c-b0ab9672bfb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:08.831408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1676399-d760-4201-a95c-b0ab9672bfb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:08.840817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1676399-d760-4201-a95c-b0ab9672bfb2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:08.880871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 667/1357] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:08.888705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1676399-d760-4201-a95c-b0ab9672bfb2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:08.921031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 691/1358] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:08 2022] POST /identity/v3/projects => generated 377 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:08.924776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc0471-b81e-4d0e-aef0-06f3a4f75aaa admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:08.928181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1676399-d760-4201-a95c-b0ab9672bfb2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:08.978643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:09.083333 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5147b91-23b7-41da-b9fe-86707d88b954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:09.084407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5147b91-23b7-41da-b9fe-86707d88b954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:09.084846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5147b91-23b7-41da-b9fe-86707d88b954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:09.085429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5147b91-23b7-41da-b9fe-86707d88b954 admin admin] PATH_INFO: `/v3/projects/8f9a8ac978e346cca0f2903d433ba3eb/users/61b931932af04572b8f3a310b49eea0f/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:09.102977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc0471-b81e-4d0e-aef0-06f3a4f75aaa admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:09.109653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b00a77-16c6-41da-9d48-58e7a798851a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:09.110150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b00a77-16c6-41da-9d48-58e7a798851a admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:09.110479 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b00a77-16c6-41da-9d48-58e7a798851a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:09.110751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b00a77-16c6-41da-9d48-58e7a798851a admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:09.111394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b00a77-16c6-41da-9d48-58e7a798851a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:09.113511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b00a77-16c6-41da-9d48-58e7a798851a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:09.127781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5147b91-23b7-41da-b9fe-86707d88b954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9a8ac978e346cca0f2903d433ba3eb, user_id=61b931932af04572b8f3a310b49eea0f, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:09.131348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5147b91-23b7-41da-b9fe-86707d88b954 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:09.144301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 692/1359] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:08 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:09.150240 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b00a77-16c6-41da-9d48-58e7a798851a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:09.161256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 668/1360] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:08 2022] PUT /identity/v3/projects/8f9a8ac978e346cca0f2903d433ba3eb/users/61b931932af04572b8f3a310b49eea0f/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:09.165277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b00a77-16c6-41da-9d48-58e7a798851a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:09.165602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5147b91-23b7-41da-b9fe-86707d88b954 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:09.176724 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:09.312184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a7794-e455-4b6c-8e43-0d8664551209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:09.312815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a7794-e455-4b6c-8e43-0d8664551209 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:09.313373 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a7794-e455-4b6c-8e43-0d8664551209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:09.313651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a7794-e455-4b6c-8e43-0d8664551209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:09.315801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5147b91-23b7-41da-b9fe-86707d88b954 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:09.316882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a7794-e455-4b6c-8e43-0d8664551209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:09.321668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a7794-e455-4b6c-8e43-0d8664551209 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:09.322129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be70410-ec49-44e2-9621-9607deb6261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:09.322874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be70410-ec49-44e2-9621-9607deb6261d admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:09.323406 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be70410-ec49-44e2-9621-9607deb6261d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:09.323780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be70410-ec49-44e2-9621-9607deb6261d admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:09.324569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be70410-ec49-44e2-9621-9607deb6261d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:09.326897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be70410-ec49-44e2-9621-9607deb6261d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:09.341537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 669/1361] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:55:09.345828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be70410-ec49-44e2-9621-9607deb6261d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:09.359365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 693/1362] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:09.363047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a7794-e455-4b6c-8e43-0d8664551209 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:09.371905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be70410-ec49-44e2-9621-9607deb6261d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:09.382670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db120d8c-1f5b-4d26-9dda-66471712c3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:09.384500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db120d8c-1f5b-4d26-9dda-66471712c3cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:09.384777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db120d8c-1f5b-4d26-9dda-66471712c3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:09.385052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db120d8c-1f5b-4d26-9dda-66471712c3cf admin admin] PATH_INFO: `/v3/projects/8f9a8ac978e346cca0f2903d433ba3eb/users/61b931932af04572b8f3a310b49eea0f/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:09.391971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db120d8c-1f5b-4d26-9dda-66471712c3cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9a8ac978e346cca0f2903d433ba3eb, user_id=61b931932af04572b8f3a310b49eea0f, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:09.394651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db120d8c-1f5b-4d26-9dda-66471712c3cf admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:09.397485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a7794-e455-4b6c-8e43-0d8664551209 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:09.413603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60ce4c-6f28-467f-8578-54871a430256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:09.416167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60ce4c-6f28-467f-8578-54871a430256 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:09.416456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60ce4c-6f28-467f-8578-54871a430256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:09.417654 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60ce4c-6f28-467f-8578-54871a430256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:09.418336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60ce4c-6f28-467f-8578-54871a430256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:09.427066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60ce4c-6f28-467f-8578-54871a430256 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:09.442524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 670/1363] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:09 2022] PUT /identity/v3/projects/8f9a8ac978e346cca0f2903d433ba3eb/users/61b931932af04572b8f3a310b49eea0f/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:09.444113 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3734feb-18b4-4703-b9ae-278acfe4bd05 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:09.444402 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3734feb-18b4-4703-b9ae-278acfe4bd05 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:09.444675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3734feb-18b4-4703-b9ae-278acfe4bd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:09.460824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 694/1364] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:55:09.465357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60ce4c-6f28-467f-8578-54871a430256 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:09.511385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60ce4c-6f28-467f-8578-54871a430256 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:09.521034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-e3734feb-18b4-4703-b9ae-278acfe4bd05 None None] MFA Rules not processed for user `61b931932af04572b8f3a310b49eea0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:09.522344 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e3734feb-18b4-4703-b9ae-278acfe4bd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:09.634191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fb1eb-f400-4215-a0d9-c570438dfc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:09.635580 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fb1eb-f400-4215-a0d9-c570438dfc92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:09.635905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fb1eb-f400-4215-a0d9-c570438dfc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:09.636174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fb1eb-f400-4215-a0d9-c570438dfc92 admin admin] PATH_INFO: `/v3/projects/e1e5f99ad67d454d904be9dc08c0411b/users/7268d20c2b6f4f24b5c03bb57dcc6a0b/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:09.678518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263fb1eb-f400-4215-a0d9-c570438dfc92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e5f99ad67d454d904be9dc08c0411b, user_id=7268d20c2b6f4f24b5c03bb57dcc6a0b, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:09.684142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263fb1eb-f400-4215-a0d9-c570438dfc92 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:09.685741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 671/1365] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:09.687321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc9d4-ee27-4fce-80b5-e88db2a42bb3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:09.687614 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc9d4-ee27-4fce-80b5-e88db2a42bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:09.687905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc9d4-ee27-4fce-80b5-e88db2a42bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:09.741597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 695/1366] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:09 2022] PUT /identity/v3/projects/e1e5f99ad67d454d904be9dc08c0411b/users/7268d20c2b6f4f24b5c03bb57dcc6a0b/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:09.743341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fb1eb-f400-4215-a0d9-c570438dfc92 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:09.846587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fb1eb-f400-4215-a0d9-c570438dfc92 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:09.853522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0b9cb-1853-48c1-8905-8a4ae881d50c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:09.854549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0b9cb-1853-48c1-8905-8a4ae881d50c service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:09.855066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0b9cb-1853-48c1-8905-8a4ae881d50c service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:09.855542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0b9cb-1853-48c1-8905-8a4ae881d50c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:09.905297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:09.915122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:09.935700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-d64bc9d4-ee27-4fce-80b5-e88db2a42bb3 None None] MFA Rules not processed for user `762e1e38e4234803991187db9b870b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:09.938207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d64bc9d4-ee27-4fce-80b5-e88db2a42bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:10.008631 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0b9cb-1853-48c1-8905-8a4ae881d50c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:10.011783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0b9cb-1853-48c1-8905-8a4ae881d50c service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:10.062981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 696/1367] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:10.065986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd9857-f569-4482-926e-4171cc34ae4d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:10.066241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd9857-f569-4482-926e-4171cc34ae4d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:10.066513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd9857-f569-4482-926e-4171cc34ae4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:10.104069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 672/1368] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:10.111811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bc9d4-ee27-4fce-80b5-e88db2a42bb3 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:10.121338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d64bc9d4-ee27-4fce-80b5-e88db2a42bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:10.165690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-24bd9857-f569-4482-926e-4171cc34ae4d None None] MFA Rules not processed for user `7268d20c2b6f4f24b5c03bb57dcc6a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:10.168191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-24bd9857-f569-4482-926e-4171cc34ae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:10.310480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bc9d4-ee27-4fce-80b5-e88db2a42bb3 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:10.325233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 697/1369] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:10.325233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd9857-f569-4482-926e-4171cc34ae4d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:10.327618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:10.328515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:10.329489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-24bd9857-f569-4482-926e-4171cc34ae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:10.329665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:10.330133 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] PATH_INFO: `/v3/users/bf96e2a560ae495e89da872a4f676c78` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:10.331454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf96e2a560ae495e89da872a4f676c78)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:10.334373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:10.490107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf96e2a560ae495e89da872a4f676c78'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:10.491064 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf96e2a560ae495e89da872a4f676c78'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:10.534567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf96e2a560ae495e89da872a4f676c78'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:10.536632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf96e2a560ae495e89da872a4f676c78'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:10.541027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd9857-f569-4482-926e-4171cc34ae4d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:10.558159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096b270-2ecd-4269-b7ba-4c9d768dcebb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:10.559919 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096b270-2ecd-4269-b7ba-4c9d768dcebb service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:10.560546 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096b270-2ecd-4269-b7ba-4c9d768dcebb service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:10.560546 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096b270-2ecd-4269-b7ba-4c9d768dcebb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:10.563200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a096b270-2ecd-4269-b7ba-4c9d768dcebb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:10.575249 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf96e2a560ae495e89da872a4f676c78'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:10.609558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096b270-2ecd-4269-b7ba-4c9d768dcebb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:10.611295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096b270-2ecd-4269-b7ba-4c9d768dcebb service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:10.622487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 673/1370] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:10 2022] DELETE /identity/v3/users/bf96e2a560ae495e89da872a4f676c78 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:10.622487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:10.628711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:10.637304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 698/1371] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:10.638189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096b270-2ecd-4269-b7ba-4c9d768dcebb service nova] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:10.642071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a096b270-2ecd-4269-b7ba-4c9d768dcebb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:10.677264 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096b270-2ecd-4269-b7ba-4c9d768dcebb service nova] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:10.736321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:10.797462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:10.831100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd264f1-7bed-4504-8881-65d5c8b55024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:10.831992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd264f1-7bed-4504-8881-65d5c8b55024 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:10.832442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd264f1-7bed-4504-8881-65d5c8b55024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:10.832442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd264f1-7bed-4504-8881-65d5c8b55024 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:10.833087 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd264f1-7bed-4504-8881-65d5c8b55024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:10.838313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd264f1-7bed-4504-8881-65d5c8b55024 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:10.909756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 699/1372] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:10 2022] POST /identity/v3/projects => generated 377 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:10.910760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd264f1-7bed-4504-8881-65d5c8b55024 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:10.914818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73a1cd-0c3b-41fc-aac7-cf37675c867b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:11.108613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd264f1-7bed-4504-8881-65d5c8b55024 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:11.109342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:11.109552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:11.109851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:11.110676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] PATH_INFO: `/v3/users/e9b6594a78e240fc9bbb50babd7a1c1a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:11.113116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b6594a78e240fc9bbb50babd7a1c1a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:11.119514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5895f4f-b50e-4dda-a427-58ca2c98c4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:11.120220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5895f4f-b50e-4dda-a427-58ca2c98c4dd admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:11.120588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5895f4f-b50e-4dda-a427-58ca2c98c4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:11.120746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5895f4f-b50e-4dda-a427-58ca2c98c4dd admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:11.121034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:11.129599 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5895f4f-b50e-4dda-a427-58ca2c98c4dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:11.135070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5895f4f-b50e-4dda-a427-58ca2c98c4dd admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:11.193186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 700/1373] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:10 2022] POST /identity/v3/users => generated 295 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:11.194474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5895f4f-b50e-4dda-a427-58ca2c98c4dd admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:11.220858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5895f4f-b50e-4dda-a427-58ca2c98c4dd admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:11.232831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512cd89-7564-44b1-af40-0a181ca16bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:11.233474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512cd89-7564-44b1-af40-0a181ca16bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:11.233772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512cd89-7564-44b1-af40-0a181ca16bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:11.234184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512cd89-7564-44b1-af40-0a181ca16bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:11.235728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512cd89-7564-44b1-af40-0a181ca16bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:11.237193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512cd89-7564-44b1-af40-0a181ca16bd1 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:11.276442 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b6594a78e240fc9bbb50babd7a1c1a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:11.277415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b6594a78e240fc9bbb50babd7a1c1a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:11.277575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 701/1374] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:11.278891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512cd89-7564-44b1-af40-0a181ca16bd1 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:11.296869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b6594a78e240fc9bbb50babd7a1c1a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:11.298217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b6594a78e240fc9bbb50babd7a1c1a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:11.303681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512cd89-7564-44b1-af40-0a181ca16bd1 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:11.316210 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74e414-e56b-46b7-9d66-0d74986b05e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:11.317158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e414-e56b-46b7-9d66-0d74986b05e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:11.317601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e414-e56b-46b7-9d66-0d74986b05e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:11.318859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e414-e56b-46b7-9d66-0d74986b05e6 admin admin] PATH_INFO: `/v3/projects/3033d08dbe484d0a95dcaa46525cedf8/users/73a24990b57c4e42892c1e590b3073e9/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:11.328328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b6594a78e240fc9bbb50babd7a1c1a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:11.357686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 674/1375] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:10 2022] DELETE /identity/v3/users/e9b6594a78e240fc9bbb50babd7a1c1a => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:11.360374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74e414-e56b-46b7-9d66-0d74986b05e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3033d08dbe484d0a95dcaa46525cedf8, user_id=73a24990b57c4e42892c1e590b3073e9, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:11.361366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:11.364317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74e414-e56b-46b7-9d66-0d74986b05e6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:11.365385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:11.408505 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 702/1376] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:11 2022] PUT /identity/v3/projects/3033d08dbe484d0a95dcaa46525cedf8/users/73a24990b57c4e42892c1e590b3073e9/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:11.410158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c25d99-c9af-48e2-a1b9-1ba76055c2e7 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:11.412821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c25d99-c9af-48e2-a1b9-1ba76055c2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:11.413277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c25d99-c9af-48e2-a1b9-1ba76055c2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:11.457428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f7c25d99-c9af-48e2-a1b9-1ba76055c2e7 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:11.461627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f7c25d99-c9af-48e2-a1b9-1ba76055c2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:11.517532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:11.564546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba443aa-0dc0-48e8-b8dd-494cf09f8d39 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:11.653598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 703/1377] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:11.655320 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c25d99-c9af-48e2-a1b9-1ba76055c2e7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:11.659549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f7c25d99-c9af-48e2-a1b9-1ba76055c2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:11.722088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c25d99-c9af-48e2-a1b9-1ba76055c2e7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:11.736334 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:11.737607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:11.737607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:11.737607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] PATH_INFO: `/v3/projects/eacbbec0ccbb4ef097d046564d61d468` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:11.738258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ffb155-6723-4663-9fa5-9b8daa76d97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:11.739231 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffb155-6723-4663-9fa5-9b8daa76d97f admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:11.739924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eacbbec0ccbb4ef097d046564d61d468)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:11.740894 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffb155-6723-4663-9fa5-9b8daa76d97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:11.741991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:11.744632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffb155-6723-4663-9fa5-9b8daa76d97f admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:11.746759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ffb155-6723-4663-9fa5-9b8daa76d97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:11.753474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ffb155-6723-4663-9fa5-9b8daa76d97f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:11.799207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 704/1378] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:11.803748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ffb155-6723-4663-9fa5-9b8daa76d97f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:11.810304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d0ffb155-6723-4663-9fa5-9b8daa76d97f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:11.859291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:11.860289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eacbbec0ccbb4ef097d046564d61d468', 'request_id': 'req-d3406774-2b13-4ae9-9106-53ab4c8a52b1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:11.879209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eacbbec0ccbb4ef097d046564d61d468', 'request_id': 'req-d3406774-2b13-4ae9-9106-53ab4c8a52b1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:11.901758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eacbbec0ccbb4ef097d046564d61d468', 'request_id': 'req-d3406774-2b13-4ae9-9106-53ab4c8a52b1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:11.944222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] The token cache is being invalidate because project eacbbec0ccbb4ef097d046564d61d468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:11.950025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:11.957907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:11.960462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 675/1379] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:11 2022] DELETE /identity/v3/projects/eacbbec0ccbb4ef097d046564d61d468 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:11.962198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:11.969272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:11.994886 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ffb155-6723-4663-9fa5-9b8daa76d97f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:12.111452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3406774-2b13-4ae9-9106-53ab4c8a52b1 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:12.119463 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0df8-f8f5-41e5-8b4e-20b1074c4970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:12.120517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0df8-f8f5-41e5-8b4e-20b1074c4970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:12.120767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0df8-f8f5-41e5-8b4e-20b1074c4970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:12.121104 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0df8-f8f5-41e5-8b4e-20b1074c4970 admin admin] PATH_INFO: `/v3/projects/3033d08dbe484d0a95dcaa46525cedf8/users/73a24990b57c4e42892c1e590b3073e9/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:12.123392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf0df8-f8f5-41e5-8b4e-20b1074c4970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3033d08dbe484d0a95dcaa46525cedf8, user_id=73a24990b57c4e42892c1e590b3073e9, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:12.127059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf0df8-f8f5-41e5-8b4e-20b1074c4970 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:12.143439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 676/1380] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:11 2022] PUT /identity/v3/projects/3033d08dbe484d0a95dcaa46525cedf8/users/73a24990b57c4e42892c1e590b3073e9/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:12.154486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98876bd4-88ec-4d66-9a3c-72c5773dc69d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:12.155193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876bd4-88ec-4d66-9a3c-72c5773dc69d service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:12.155477 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876bd4-88ec-4d66-9a3c-72c5773dc69d service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:12.155597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0df8-f8f5-41e5-8b4e-20b1074c4970 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:12.155741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876bd4-88ec-4d66-9a3c-72c5773dc69d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:12.157716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-24cf0df8-f8f5-41e5-8b4e-20b1074c4970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:12.158084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-98876bd4-88ec-4d66-9a3c-72c5773dc69d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:12.183381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876bd4-88ec-4d66-9a3c-72c5773dc69d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:12.184927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876bd4-88ec-4d66-9a3c-72c5773dc69d service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:12.193884 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:12.211648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 705/1381] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:12.217312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848994a8-0d98-4afd-98bd-b8954494c684 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:12.217598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848994a8-0d98-4afd-98bd-b8954494c684 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:12.217872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848994a8-0d98-4afd-98bd-b8954494c684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:12.275720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-848994a8-0d98-4afd-98bd-b8954494c684 None None] MFA Rules not processed for user `73a24990b57c4e42892c1e590b3073e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:12.277071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-848994a8-0d98-4afd-98bd-b8954494c684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:12.308950 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0df8-f8f5-41e5-8b4e-20b1074c4970 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:12.316234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:12.316898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:12.317330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:12.317691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] PATH_INFO: `/v3/users/3597d46e97d6497fa758bb0dab9d5743` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:12.318921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3597d46e97d6497fa758bb0dab9d5743)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:12.321851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:12.439564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3597d46e97d6497fa758bb0dab9d5743'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:12.444489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3597d46e97d6497fa758bb0dab9d5743'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:12.463329 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 706/1382] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:12.470429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848994a8-0d98-4afd-98bd-b8954494c684 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:12.472715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-848994a8-0d98-4afd-98bd-b8954494c684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:12.474056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3597d46e97d6497fa758bb0dab9d5743'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:12.478413 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3597d46e97d6497fa758bb0dab9d5743'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:12.521552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3597d46e97d6497fa758bb0dab9d5743'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:12.546307 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 677/1383] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:12 2022] DELETE /identity/v3/users/3597d46e97d6497fa758bb0dab9d5743 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:12.645107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848994a8-0d98-4afd-98bd-b8954494c684 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:12.672433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:12.755254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638aeb8d-a459-4fd8-8c3b-ec33e2d06dd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:12.755854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638aeb8d-a459-4fd8-8c3b-ec33e2d06dd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:12.756156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638aeb8d-a459-4fd8-8c3b-ec33e2d06dd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:12.756420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638aeb8d-a459-4fd8-8c3b-ec33e2d06dd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:12.768857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-638aeb8d-a459-4fd8-8c3b-ec33e2d06dd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:12.909198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638aeb8d-a459-4fd8-8c3b-ec33e2d06dd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:12.910745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638aeb8d-a459-4fd8-8c3b-ec33e2d06dd3 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:12.934648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 707/1384] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:55:13.085779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:13.088103 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:13.126895 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bda1af-c3d2-4db4-9904-f299fd3199ca admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:13.143984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbee0f-7937-4501-afb3-ec5512331c72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:13.144616 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbee0f-7937-4501-afb3-ec5512331c72 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:13.144889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbee0f-7937-4501-afb3-ec5512331c72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:13.145156 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbee0f-7937-4501-afb3-ec5512331c72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:13.155905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7fdbee0f-7937-4501-afb3-ec5512331c72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:13.263304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbee0f-7937-4501-afb3-ec5512331c72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:13.267381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbee0f-7937-4501-afb3-ec5512331c72 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:13.314632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 678/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:13.603598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638aeb8d-a459-4fd8-8c3b-ec33e2d06dd3 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:13.606043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-638aeb8d-a459-4fd8-8c3b-ec33e2d06dd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:13.839602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638aeb8d-a459-4fd8-8c3b-ec33e2d06dd3 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:13.852218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:13.853128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:13.853668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:13.854072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] PATH_INFO: `/v3/projects/50048b7509fe4bc89b4cff10cacc4681` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:13.856076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50048b7509fe4bc89b4cff10cacc4681)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:13.860261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:13.972456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50048b7509fe4bc89b4cff10cacc4681', 'request_id': 'req-0f6fffd6-85db-404b-bff7-321787fa8994'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:13.983344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50048b7509fe4bc89b4cff10cacc4681', 'request_id': 'req-0f6fffd6-85db-404b-bff7-321787fa8994'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.000813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50048b7509fe4bc89b4cff10cacc4681', 'request_id': 'req-0f6fffd6-85db-404b-bff7-321787fa8994'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.009358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:14.033091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] The token cache is being invalidate because project 50048b7509fe4bc89b4cff10cacc4681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:14.034624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.036221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0f6fffd6-85db-404b-bff7-321787fa8994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.048394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 708/1386] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:13 2022] DELETE /identity/v3/projects/50048b7509fe4bc89b4cff10cacc4681 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:14.048955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8e552-670f-49d1-a66a-30811252fb11 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:14.048955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8e552-670f-49d1-a66a-30811252fb11 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:14.048955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8e552-670f-49d1-a66a-30811252fb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:14.050627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323c291-916f-449d-9109-ebc950fb8c48 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:14.051024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323c291-916f-449d-9109-ebc950fb8c48 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:14.051395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323c291-916f-449d-9109-ebc950fb8c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:14.081398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a7b8e552-670f-49d1-a66a-30811252fb11 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:14.082617 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a7b8e552-670f-49d1-a66a-30811252fb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:14.118551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8323c291-916f-449d-9109-ebc950fb8c48 None None] MFA Rules not processed for user `07441f3b8a3841b0ba3677ec38fe7f32`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:14.120210 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8323c291-916f-449d-9109-ebc950fb8c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:14.153297 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:14.229492 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 709/1387] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:14.230442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323c291-916f-449d-9109-ebc950fb8c48 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:14.232541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8323c291-916f-449d-9109-ebc950fb8c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:14.251375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 679/1388] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:14.252807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555fde7-072e-42c5-90e5-30edc1611522 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:14.253090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555fde7-072e-42c5-90e5-30edc1611522 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:14.253360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555fde7-072e-42c5-90e5-30edc1611522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:14.264685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323c291-916f-449d-9109-ebc950fb8c48 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:14.292512 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:14.293358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:14.293745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:14.294120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] PATH_INFO: `/v3/projects/28c0ea32e702484593f155ceb4d4b3c0` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:14.295390 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=28c0ea32e702484593f155ceb4d4b3c0)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:14.295585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2555fde7-072e-42c5-90e5-30edc1611522 None None] MFA Rules not processed for user `07441f3b8a3841b0ba3677ec38fe7f32`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:14.296860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2555fde7-072e-42c5-90e5-30edc1611522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:14.299638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:14.323148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 680/1389] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:14.342239 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfa0a5-8626-4081-9682-ba87944fc287 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:14.343195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfa0a5-8626-4081-9682-ba87944fc287 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:14.343467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfa0a5-8626-4081-9682-ba87944fc287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:14.404980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-3dcfa0a5-8626-4081-9682-ba87944fc287 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:14.407263 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3dcfa0a5-8626-4081-9682-ba87944fc287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:14.425996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28c0ea32e702484593f155ceb4d4b3c0', 'request_id': 'req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.442145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28c0ea32e702484593f155ceb4d4b3c0', 'request_id': 'req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.467470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28c0ea32e702484593f155ceb4d4b3c0', 'request_id': 'req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.491814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] The token cache is being invalidate because project 28c0ea32e702484593f155ceb4d4b3c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:14.493033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.493865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.495796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 710/1390] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:14 2022] DELETE /identity/v3/projects/28c0ea32e702484593f155ceb4d4b3c0 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:14.500369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:14.504252 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:14.608583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 681/1391] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:14.610062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfa0a5-8626-4081-9682-ba87944fc287 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:14.613625 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3dcfa0a5-8626-4081-9682-ba87944fc287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:14.627158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:14.642347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a02f69-716c-4fce-9eb1-76afa1a4cc1f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:14.646269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfa0a5-8626-4081-9682-ba87944fc287 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:14.658867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7c8d4-3811-4c95-a573-fd127d0d64bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:14.665950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7c8d4-3811-4c95-a573-fd127d0d64bc service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:14.666281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7c8d4-3811-4c95-a573-fd127d0d64bc service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:14.666529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7c8d4-3811-4c95-a573-fd127d0d64bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:14.667793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:14.668470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:14.668738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:14.669005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] PATH_INFO: `/v3/users/61b931932af04572b8f3a310b49eea0f` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:14.670504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b931932af04572b8f3a310b49eea0f)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:14.672478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:14.674052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2cc7c8d4-3811-4c95-a573-fd127d0d64bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:14.842121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b931932af04572b8f3a310b49eea0f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.845370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b931932af04572b8f3a310b49eea0f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.872935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b931932af04572b8f3a310b49eea0f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.879930 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b931932af04572b8f3a310b49eea0f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.913270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7c8d4-3811-4c95-a573-fd127d0d64bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:14.923679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7c8d4-3811-4c95-a573-fd127d0d64bc service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:14.942504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-853cfa81-2db2-466b-93aa-53550a606a6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b931932af04572b8f3a310b49eea0f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:14.968255 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 711/1392] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:14.971261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7c8d4-3811-4c95-a573-fd127d0d64bc service nova] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:14.999089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 682/1393] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:14 2022] DELETE /identity/v3/users/61b931932af04572b8f3a310b49eea0f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:15.016863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7c8d4-3811-4c95-a573-fd127d0d64bc service nova] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:15.152325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcea34b-7870-474e-84c6-17fdde714265 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:15.152895 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcea34b-7870-474e-84c6-17fdde714265 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:15.153356 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcea34b-7870-474e-84c6-17fdde714265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:15.234853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-5dcea34b-7870-474e-84c6-17fdde714265 None None] MFA Rules not processed for user `c091bf027839447da4c713deb291ed1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:15.237101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5dcea34b-7870-474e-84c6-17fdde714265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:15.277405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672cd6e-1c66-4361-98b1-256c84f407e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:15.278301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672cd6e-1c66-4361-98b1-256c84f407e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:15.280142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672cd6e-1c66-4361-98b1-256c84f407e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:15.280784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672cd6e-1c66-4361-98b1-256c84f407e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:15.283748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672cd6e-1c66-4361-98b1-256c84f407e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:15.284816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672cd6e-1c66-4361-98b1-256c84f407e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:15.290776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672cd6e-1c66-4361-98b1-256c84f407e1 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:15.334226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 712/1394] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:15.339129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672cd6e-1c66-4361-98b1-256c84f407e1 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:15.343460 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e672cd6e-1c66-4361-98b1-256c84f407e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:15.363469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:15.402960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672cd6e-1c66-4361-98b1-256c84f407e1 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:15.412876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f7539-d4b3-4ac6-97f0-5026a706bba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:15.413496 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f7539-d4b3-4ac6-97f0-5026a706bba8 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:15.413769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f7539-d4b3-4ac6-97f0-5026a706bba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:15.414634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f7539-d4b3-4ac6-97f0-5026a706bba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:15.415212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f7539-d4b3-4ac6-97f0-5026a706bba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:15.431619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f7539-d4b3-4ac6-97f0-5026a706bba8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:15.494347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 683/1395] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:15.506454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36e6c-8e91-4ce2-a7b2-4626ebdf6530 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:15.507130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36e6c-8e91-4ce2-a7b2-4626ebdf6530 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:15.507741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36e6c-8e91-4ce2-a7b2-4626ebdf6530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:15.525581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 713/1396] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:15 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:15.528893 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f7539-d4b3-4ac6-97f0-5026a706bba8 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:15.556695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a3b36e6c-8e91-4ce2-a7b2-4626ebdf6530 None None] MFA Rules not processed for user `c091bf027839447da4c713deb291ed1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:15.558003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a3b36e6c-8e91-4ce2-a7b2-4626ebdf6530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:15.718203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f7539-d4b3-4ac6-97f0-5026a706bba8 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:15.724908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b594d5-8f26-45d9-abec-35923f7dd856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:15.725663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b594d5-8f26-45d9-abec-35923f7dd856 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:15.725944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b594d5-8f26-45d9-abec-35923f7dd856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:15.727008 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b594d5-8f26-45d9-abec-35923f7dd856 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:15.728730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b594d5-8f26-45d9-abec-35923f7dd856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:15.732314 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b594d5-8f26-45d9-abec-35923f7dd856 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:15.751575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 684/1397] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:15.755894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df7cc4-cf5d-49f0-b779-5609276c6c9a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:15.755894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df7cc4-cf5d-49f0-b779-5609276c6c9a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:15.756640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df7cc4-cf5d-49f0-b779-5609276c6c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:15.805793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 714/1398] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:15 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:15.805793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b594d5-8f26-45d9-abec-35923f7dd856 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:15.805793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-09b594d5-8f26-45d9-abec-35923f7dd856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:15.805793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:15.842253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-35df7cc4-cf5d-49f0-b779-5609276c6c9a None None] MFA Rules not processed for user `b9187955c3cf4b77ba86e67a6f378e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:15.843590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-35df7cc4-cf5d-49f0-b779-5609276c6c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:15.981438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b594d5-8f26-45d9-abec-35923f7dd856 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:15.992273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e726f7-e01d-4b39-a1f3-8ad2f69719f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:15.993408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e726f7-e01d-4b39-a1f3-8ad2f69719f8 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:15.994322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e726f7-e01d-4b39-a1f3-8ad2f69719f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:15.994322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e726f7-e01d-4b39-a1f3-8ad2f69719f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.012802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 685/1399] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:16.016277 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df7cc4-cf5d-49f0-b779-5609276c6c9a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:16.021175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e726f7-e01d-4b39-a1f3-8ad2f69719f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:16.027496 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e726f7-e01d-4b39-a1f3-8ad2f69719f8 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:16.036673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df7cc4-cf5d-49f0-b779-5609276c6c9a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:16.041613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 715/1400] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:16.043224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db4909-e4c7-476b-b1bf-ce53e7fbffa7 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.043508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db4909-e4c7-476b-b1bf-ce53e7fbffa7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.043780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db4909-e4c7-476b-b1bf-ce53e7fbffa7 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.052293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 716/1401] 10.222.0.27 () {54 vars in 912 bytes} [Fri Oct 14 17:55:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:55:16.054055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef52618-e4b5-4638-b865-4fc81677897c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.054184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef52618-e4b5-4638-b865-4fc81677897c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.054410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef52618-e4b5-4638-b865-4fc81677897c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.060472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100b618-388a-4af9-add3-324ae82a6816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:16.061539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100b618-388a-4af9-add3-324ae82a6816 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.062614 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100b618-388a-4af9-add3-324ae82a6816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.064309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100b618-388a-4af9-add3-324ae82a6816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.065199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100b618-388a-4af9-add3-324ae82a6816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:16.071803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100b618-388a-4af9-add3-324ae82a6816 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:16.093042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 686/1402] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:16.097284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100b618-388a-4af9-add3-324ae82a6816 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:16.112541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0ef52618-e4b5-4638-b865-4fc81677897c None None] MFA Rules not processed for user `3c17f274e4c14f2f99650844642437a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:16.114144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0ef52618-e4b5-4638-b865-4fc81677897c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:16.126591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100b618-388a-4af9-add3-324ae82a6816 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:16.140810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba7f01-669c-415f-9dee-d06b4d76e855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:16.142011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba7f01-669c-415f-9dee-d06b4d76e855 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.144283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba7f01-669c-415f-9dee-d06b4d76e855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.150022 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba7f01-669c-415f-9dee-d06b4d76e855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.150872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba7f01-669c-415f-9dee-d06b4d76e855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:16.157510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba7f01-669c-415f-9dee-d06b4d76e855 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:16.189805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 687/1403] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:16.193855 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba7f01-669c-415f-9dee-d06b4d76e855 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:16.210466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba7f01-669c-415f-9dee-d06b4d76e855 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:16.216662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd28ace-d12d-48e1-911e-ddd8ed4b2a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:16.217967 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd28ace-d12d-48e1-911e-ddd8ed4b2a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.218347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd28ace-d12d-48e1-911e-ddd8ed4b2a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.218750 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd28ace-d12d-48e1-911e-ddd8ed4b2a9c admin admin] PATH_INFO: `/v3/projects/069be3feed4a45bbb848710c19d28052/users/acefc47dc32044e9a52f7fbedd1293c8/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.259376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 717/1404] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:55:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:16.260924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef52618-e4b5-4638-b865-4fc81677897c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:16.262554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0ef52618-e4b5-4638-b865-4fc81677897c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:16.282725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd28ace-d12d-48e1-911e-ddd8ed4b2a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=069be3feed4a45bbb848710c19d28052, user_id=acefc47dc32044e9a52f7fbedd1293c8, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:16.286179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd28ace-d12d-48e1-911e-ddd8ed4b2a9c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:16.302430 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef52618-e4b5-4638-b865-4fc81677897c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:16.311323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 688/1405] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:16 2022] PUT /identity/v3/projects/069be3feed4a45bbb848710c19d28052/users/acefc47dc32044e9a52f7fbedd1293c8/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:16.321774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd28ace-d12d-48e1-911e-ddd8ed4b2a9c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:16.323970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8fd28ace-d12d-48e1-911e-ddd8ed4b2a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:16.368952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:16.425704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd28ace-d12d-48e1-911e-ddd8ed4b2a9c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:16.430916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7fb2d-4d3b-4bbe-9aac-f11dbb9af8f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:16.431805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7fb2d-4d3b-4bbe-9aac-f11dbb9af8f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.432324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7fb2d-4d3b-4bbe-9aac-f11dbb9af8f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.432721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7fb2d-4d3b-4bbe-9aac-f11dbb9af8f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.437677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:16.438681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.439118 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.440311 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] PATH_INFO: `/v3/projects/8f9a8ac978e346cca0f2903d433ba3eb` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.442470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9a8ac978e346cca0f2903d433ba3eb)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:16.447543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:16.451820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7fb2d-4d3b-4bbe-9aac-f11dbb9af8f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:16.453468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7fb2d-4d3b-4bbe-9aac-f11dbb9af8f0 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:16.479984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 689/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:55:16.482447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926cfd2-f64d-41b5-9359-721f62f3939a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.482447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926cfd2-f64d-41b5-9359-721f62f3939a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.484864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926cfd2-f64d-41b5-9359-721f62f3939a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.563630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f926cfd2-f64d-41b5-9359-721f62f3939a None None] MFA Rules not processed for user `acefc47dc32044e9a52f7fbedd1293c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:16.569008 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f926cfd2-f64d-41b5-9359-721f62f3939a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:16.636759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:16.675063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9a8ac978e346cca0f2903d433ba3eb', 'request_id': 'req-b3fbda9c-c0c3-4564-9e48-c8af4121c157'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:16.688613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9a8ac978e346cca0f2903d433ba3eb', 'request_id': 'req-b3fbda9c-c0c3-4564-9e48-c8af4121c157'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:16.706934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9a8ac978e346cca0f2903d433ba3eb', 'request_id': 'req-b3fbda9c-c0c3-4564-9e48-c8af4121c157'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:16.737817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] The token cache is being invalidate because project 8f9a8ac978e346cca0f2903d433ba3eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:16.738117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:16.738928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b3fbda9c-c0c3-4564-9e48-c8af4121c157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:16.741404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 690/1407] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:16.746520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926cfd2-f64d-41b5-9359-721f62f3939a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:16.753119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 718/1408] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:16 2022] DELETE /identity/v3/projects/8f9a8ac978e346cca0f2903d433ba3eb => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:16.759738 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe1d75-9a1f-4329-8771-36d5e3c3bfe1 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.760462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe1d75-9a1f-4329-8771-36d5e3c3bfe1 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.760843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe1d75-9a1f-4329-8771-36d5e3c3bfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.780246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926cfd2-f64d-41b5-9359-721f62f3939a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:16.825594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-03fe1d75-9a1f-4329-8771-36d5e3c3bfe1 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:16.835082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-03fe1d75-9a1f-4329-8771-36d5e3c3bfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:16.926230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa0781-7578-4c03-b2db-130a619b033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:16.927314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa0781-7578-4c03-b2db-130a619b033e admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.933741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa0781-7578-4c03-b2db-130a619b033e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.934255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa0781-7578-4c03-b2db-130a619b033e admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.935642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa0781-7578-4c03-b2db-130a619b033e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:16.938215 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa0781-7578-4c03-b2db-130a619b033e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:16.970024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 719/1409] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:16.978689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1331-c6ef-4fc2-b9f9-cae10850f4da None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.979005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1331-c6ef-4fc2-b9f9-cae10850f4da None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.979276 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1331-c6ef-4fc2-b9f9-cae10850f4da None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.985718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 720/1410] 10.222.0.27 () {54 vars in 912 bytes} [Fri Oct 14 17:55:16 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:55:16.987610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae12410-7444-462d-9a54-71ffed110b33 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.987610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae12410-7444-462d-9a54-71ffed110b33 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.987810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae12410-7444-462d-9a54-71ffed110b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:16.992753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 691/1411] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:16 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:16.996764 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a03e4-ad47-48a2-b645-7f2e1d708465 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:16.997053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a03e4-ad47-48a2-b645-7f2e1d708465 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:16.997357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a03e4-ad47-48a2-b645-7f2e1d708465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:17.020905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-fae12410-7444-462d-9a54-71ffed110b33 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:17.023303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-fae12410-7444-462d-9a54-71ffed110b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:17.091118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ef7a03e4-ad47-48a2-b645-7f2e1d708465 None None] MFA Rules not processed for user `3c17f274e4c14f2f99650844642437a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:17.092526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ef7a03e4-ad47-48a2-b645-7f2e1d708465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:17.139355 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 721/1412] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:17.143357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae12410-7444-462d-9a54-71ffed110b33 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:17.151526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-fae12410-7444-462d-9a54-71ffed110b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:17.170823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae12410-7444-462d-9a54-71ffed110b33 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:17.182323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:17.193831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146f89f-afcb-4b1c-966d-93ccad6f7da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:17.194473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146f89f-afcb-4b1c-966d-93ccad6f7da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:17.194830 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146f89f-afcb-4b1c-966d-93ccad6f7da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:17.195206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146f89f-afcb-4b1c-966d-93ccad6f7da8 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:17.196340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146f89f-afcb-4b1c-966d-93ccad6f7da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:17.200659 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146f89f-afcb-4b1c-966d-93ccad6f7da8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:17.232022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 722/1413] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:17 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:17.232022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146f89f-afcb-4b1c-966d-93ccad6f7da8 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:17.233220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 692/1414] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:55:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:17.234179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a03e4-ad47-48a2-b645-7f2e1d708465 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:17.249524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146f89f-afcb-4b1c-966d-93ccad6f7da8 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:17.255278 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf24efb-b5c2-4457-96c0-b931eed89f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:17.256144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf24efb-b5c2-4457-96c0-b931eed89f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:17.256860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf24efb-b5c2-4457-96c0-b931eed89f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:17.257456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf24efb-b5c2-4457-96c0-b931eed89f1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:17.261043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a03e4-ad47-48a2-b645-7f2e1d708465 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:17.261528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf24efb-b5c2-4457-96c0-b931eed89f1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:17.261942 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf24efb-b5c2-4457-96c0-b931eed89f1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:17.263868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf24efb-b5c2-4457-96c0-b931eed89f1a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:17.285708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a2265-dee2-4938-9bcc-ecb2f6641add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:17.286339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a2265-dee2-4938-9bcc-ecb2f6641add admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:17.286621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a2265-dee2-4938-9bcc-ecb2f6641add admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:17.286933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a2265-dee2-4938-9bcc-ecb2f6641add admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:17.287591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a2265-dee2-4938-9bcc-ecb2f6641add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:17.298040 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 723/1415] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:17.300399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a2265-dee2-4938-9bcc-ecb2f6641add admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:17.304729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf24efb-b5c2-4457-96c0-b931eed89f1a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:17.310503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-fdf24efb-b5c2-4457-96c0-b931eed89f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:17.332759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf24efb-b5c2-4457-96c0-b931eed89f1a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:17.334430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 693/1416] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:17.335157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a2265-dee2-4938-9bcc-ecb2f6641add admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:17.349091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7079ac9-8721-4d1b-a1e8-8da9355d7c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:17.349091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7079ac9-8721-4d1b-a1e8-8da9355d7c8b service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:17.349091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7079ac9-8721-4d1b-a1e8-8da9355d7c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:17.349091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7079ac9-8721-4d1b-a1e8-8da9355d7c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:17.375392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7079ac9-8721-4d1b-a1e8-8da9355d7c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:17.377006 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7079ac9-8721-4d1b-a1e8-8da9355d7c8b service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:17.401322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a2265-dee2-4938-9bcc-ecb2f6641add admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:17.407689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 724/1417] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:17.409434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7079ac9-8721-4d1b-a1e8-8da9355d7c8b service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:17.413147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4857f-de6b-4475-bcf6-b88334c60b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:17.415171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4857f-de6b-4475-bcf6-b88334c60b7a admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:17.416087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4857f-de6b-4475-bcf6-b88334c60b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:17.416517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4857f-de6b-4475-bcf6-b88334c60b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:17.418096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4857f-de6b-4475-bcf6-b88334c60b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:17.422853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4857f-de6b-4475-bcf6-b88334c60b7a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:17.439427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7079ac9-8721-4d1b-a1e8-8da9355d7c8b service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:17.452912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a35c4b-53f3-4fcd-ac57-94bdb8d019a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:17.453626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a35c4b-53f3-4fcd-ac57-94bdb8d019a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:17.453902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a35c4b-53f3-4fcd-ac57-94bdb8d019a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:17.454173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a35c4b-53f3-4fcd-ac57-94bdb8d019a1 admin admin] PATH_INFO: `/v3/projects/35634575726d46d3a6736b3f5b25f50a/users/707685cd46974201ba709615ce483d1c/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:17.479730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:17.497384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a35c4b-53f3-4fcd-ac57-94bdb8d019a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35634575726d46d3a6736b3f5b25f50a, user_id=707685cd46974201ba709615ce483d1c, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:17.509799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 694/1418] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:17 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:17.510803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4857f-de6b-4475-bcf6-b88334c60b7a admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:17.515634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a35c4b-53f3-4fcd-ac57-94bdb8d019a1 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:17.516732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-24a4857f-de6b-4475-bcf6-b88334c60b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:17.545544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 725/1419] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:17 2022] PUT /identity/v3/projects/35634575726d46d3a6736b3f5b25f50a/users/707685cd46974201ba709615ce483d1c/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:17.546776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba603cfb-69f0-43d5-a77a-f6c458a9a5ac None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:17.548261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba603cfb-69f0-43d5-a77a-f6c458a9a5ac None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:17.548487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba603cfb-69f0-43d5-a77a-f6c458a9a5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:17.581204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ba603cfb-69f0-43d5-a77a-f6c458a9a5ac None None] MFA Rules not processed for user `762e1e38e4234803991187db9b870b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:17.582519 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ba603cfb-69f0-43d5-a77a-f6c458a9a5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:17.658996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4857f-de6b-4475-bcf6-b88334c60b7a admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:17.709865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 726/1420] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:17.709865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba603cfb-69f0-43d5-a77a-f6c458a9a5ac None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:17.869396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba603cfb-69f0-43d5-a77a-f6c458a9a5ac None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:17.880902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901587e0-895b-4b9e-bba4-1b73d620c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:17.881776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901587e0-895b-4b9e-bba4-1b73d620c20c admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:17.882129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901587e0-895b-4b9e-bba4-1b73d620c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:17.882407 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901587e0-895b-4b9e-bba4-1b73d620c20c admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:17.883091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901587e0-895b-4b9e-bba4-1b73d620c20c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:17.885406 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0ae81-f08f-45cc-9f16-8c8800bbc749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:17.886043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0ae81-f08f-45cc-9f16-8c8800bbc749 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:17.886318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0ae81-f08f-45cc-9f16-8c8800bbc749 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:17.886582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0ae81-f08f-45cc-9f16-8c8800bbc749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:17.887915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901587e0-895b-4b9e-bba4-1b73d620c20c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:17.893062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-46c0ae81-f08f-45cc-9f16-8c8800bbc749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:17.946544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 727/1421] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:17 2022] POST /identity/v3/users => generated 294 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:17.946544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901587e0-895b-4b9e-bba4-1b73d620c20c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:17.954988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-901587e0-895b-4b9e-bba4-1b73d620c20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:18.028343 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:18.046319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0ae81-f08f-45cc-9f16-8c8800bbc749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:18.048102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0ae81-f08f-45cc-9f16-8c8800bbc749 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:18.062528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 695/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:18.063623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0ae81-f08f-45cc-9f16-8c8800bbc749 service nova] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:18.083746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0ae81-f08f-45cc-9f16-8c8800bbc749 service nova] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:18.090076 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecd5aa-d0a0-4894-8c69-774a3998e585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:18.090847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd5aa-d0a0-4894-8c69-774a3998e585 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.091232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd5aa-d0a0-4894-8c69-774a3998e585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.091609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd5aa-d0a0-4894-8c69-774a3998e585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.092401 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecd5aa-d0a0-4894-8c69-774a3998e585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:18.094406 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecd5aa-d0a0-4894-8c69-774a3998e585 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:18.118648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 696/1423] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:18.119471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecd5aa-d0a0-4894-8c69-774a3998e585 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:18.128003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901587e0-895b-4b9e-bba4-1b73d620c20c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:18.138765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da533a55-6c50-46f0-a5e3-a4fb4cdb1210 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:18.139403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da533a55-6c50-46f0-a5e3-a4fb4cdb1210 service placement] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.140176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da533a55-6c50-46f0-a5e3-a4fb4cdb1210 service placement] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.140457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da533a55-6c50-46f0-a5e3-a4fb4cdb1210 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.142630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-da533a55-6c50-46f0-a5e3-a4fb4cdb1210 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:18.155735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecd5aa-d0a0-4894-8c69-774a3998e585 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:18.162352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da533a55-6c50-46f0-a5e3-a4fb4cdb1210 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:18.166262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da533a55-6c50-46f0-a5e3-a4fb4cdb1210 service placement] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:18.169510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4c7f3-5cfd-4aa1-bc05-86a47569dcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:18.170813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4c7f3-5cfd-4aa1-bc05-86a47569dcf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.178808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4c7f3-5cfd-4aa1-bc05-86a47569dcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.179252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4c7f3-5cfd-4aa1-bc05-86a47569dcf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.180603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4c7f3-5cfd-4aa1-bc05-86a47569dcf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:18.182997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4c7f3-5cfd-4aa1-bc05-86a47569dcf7 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:18.206539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 697/1424] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:18.208178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4c7f3-5cfd-4aa1-bc05-86a47569dcf7 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:18.212312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 728/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:18.219579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cddf4-551b-410b-8c9f-119f3d876414 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.222526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cddf4-551b-410b-8c9f-119f3d876414 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.223028 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cddf4-551b-410b-8c9f-119f3d876414 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.242174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 729/1426] 10.222.0.27 () {54 vars in 961 bytes} [Fri Oct 14 17:55:18 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:55:18.247019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cddf4-551b-410b-8c9f-119f3d876414 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:18.263108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4c7f3-5cfd-4aa1-bc05-86a47569dcf7 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:18.283541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d94d8d-ed78-4148-a94c-0f8e6294f192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:18.286161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d94d8d-ed78-4148-a94c-0f8e6294f192 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.286574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d94d8d-ed78-4148-a94c-0f8e6294f192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.287273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d94d8d-ed78-4148-a94c-0f8e6294f192 admin admin] PATH_INFO: `/v3/projects/35634575726d46d3a6736b3f5b25f50a/users/707685cd46974201ba709615ce483d1c/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.295684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d94d8d-ed78-4148-a94c-0f8e6294f192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35634575726d46d3a6736b3f5b25f50a, user_id=707685cd46974201ba709615ce483d1c, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:18.316158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d94d8d-ed78-4148-a94c-0f8e6294f192 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:18.325273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cddf4-551b-410b-8c9f-119f3d876414 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:18.338656 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 698/1427] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:18 2022] PUT /identity/v3/projects/35634575726d46d3a6736b3f5b25f50a/users/707685cd46974201ba709615ce483d1c/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:18.343014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3da55c-d217-4dfa-af05-229aeed23988 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.353625 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3da55c-d217-4dfa-af05-229aeed23988 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.354091 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3da55c-d217-4dfa-af05-229aeed23988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.386402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:18.419905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-fd3da55c-d217-4dfa-af05-229aeed23988 None None] MFA Rules not processed for user `73089bc9f85a405b86c4cd3f8cfa830a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:18.435987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-fd3da55c-d217-4dfa-af05-229aeed23988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:18.495829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67fb4e-6ed0-4d8e-bd6e-0780f9cfbad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:18.496525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67fb4e-6ed0-4d8e-bd6e-0780f9cfbad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.496933 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67fb4e-6ed0-4d8e-bd6e-0780f9cfbad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.497200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67fb4e-6ed0-4d8e-bd6e-0780f9cfbad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.498337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67fb4e-6ed0-4d8e-bd6e-0780f9cfbad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:18.503603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67fb4e-6ed0-4d8e-bd6e-0780f9cfbad7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:18.517843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 730/1428] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:18.519207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80e0bb-f720-427d-940a-8f8d026dd0a0 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.519551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80e0bb-f720-427d-940a-8f8d026dd0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.519912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80e0bb-f720-427d-940a-8f8d026dd0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.561289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8c80e0bb-f720-427d-940a-8f8d026dd0a0 None None] MFA Rules not processed for user `707685cd46974201ba709615ce483d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:18.562771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8c80e0bb-f720-427d-940a-8f8d026dd0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:18.627904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 699/1429] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Oct 14 17:55:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:18.636675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3da55c-d217-4dfa-af05-229aeed23988 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:18.668440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3da55c-d217-4dfa-af05-229aeed23988 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:18.682650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbf8ec-78d2-427a-b62a-fbd28db70f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:18.683344 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbf8ec-78d2-427a-b62a-fbd28db70f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.683629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbf8ec-78d2-427a-b62a-fbd28db70f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.683944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbf8ec-78d2-427a-b62a-fbd28db70f3c admin admin] PATH_INFO: `/v3/projects/17a23a40169b43db87861e5a3b0afa3f/users/f9e2b4a852f6446a8055a56fe658521c/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.733683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbf8ec-78d2-427a-b62a-fbd28db70f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a23a40169b43db87861e5a3b0afa3f, user_id=f9e2b4a852f6446a8055a56fe658521c, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:18.736549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbf8ec-78d2-427a-b62a-fbd28db70f3c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:18.753409 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 731/1430] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:18.761896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aedf0-98f4-4baa-8eb8-0d5c3e83cc72 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.762901 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aedf0-98f4-4baa-8eb8-0d5c3e83cc72 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.764358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aedf0-98f4-4baa-8eb8-0d5c3e83cc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.772321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 700/1431] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:18 2022] PUT /identity/v3/projects/17a23a40169b43db87861e5a3b0afa3f/users/f9e2b4a852f6446a8055a56fe658521c/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:18.773145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbf8ec-78d2-427a-b62a-fbd28db70f3c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:18.811540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-313aedf0-98f4-4baa-8eb8-0d5c3e83cc72 None None] MFA Rules not processed for user `c091bf027839447da4c713deb291ed1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:18.813204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-313aedf0-98f4-4baa-8eb8-0d5c3e83cc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:18.894802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 732/1432] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:18.895843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aedf0-98f4-4baa-8eb8-0d5c3e83cc72 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:18.898090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-313aedf0-98f4-4baa-8eb8-0d5c3e83cc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:18.940753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbf8ec-78d2-427a-b62a-fbd28db70f3c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:18.959363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b08e75-5a68-43e8-ba53-1e0079f6f15f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:18.963509 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08e75-5a68-43e8-ba53-1e0079f6f15f service placement] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:18.964208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08e75-5a68-43e8-ba53-1e0079f6f15f service placement] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:18.964870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08e75-5a68-43e8-ba53-1e0079f6f15f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:18.975713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:18.994078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b08e75-5a68-43e8-ba53-1e0079f6f15f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:18.996928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b08e75-5a68-43e8-ba53-1e0079f6f15f service placement] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:19.009657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aedf0-98f4-4baa-8eb8-0d5c3e83cc72 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:19.022065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6630f60-b910-4b86-bf50-bb92a35abde9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:19.023180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6630f60-b910-4b86-bf50-bb92a35abde9 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:19.023180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6630f60-b910-4b86-bf50-bb92a35abde9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:19.023580 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6630f60-b910-4b86-bf50-bb92a35abde9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:19.040608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 701/1433] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:19.047210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef245965-4bf8-467a-abbb-f194bc13f18f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:19.048808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef245965-4bf8-467a-abbb-f194bc13f18f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:19.049118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef245965-4bf8-467a-abbb-f194bc13f18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:19.158732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ef245965-4bf8-467a-abbb-f194bc13f18f None None] MFA Rules not processed for user `f9e2b4a852f6446a8055a56fe658521c`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:19.160014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ef245965-4bf8-467a-abbb-f194bc13f18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:19.172896 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:19.237342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6630f60-b910-4b86-bf50-bb92a35abde9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:19.239869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6630f60-b910-4b86-bf50-bb92a35abde9 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:19.256524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 733/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:19.416525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 702/1435] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:19.421230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6630f60-b910-4b86-bf50-bb92a35abde9 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:19.581840 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6630f60-b910-4b86-bf50-bb92a35abde9 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:19.594562 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f2f0d-b91d-4183-aacf-ed32e9b612e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:19.595353 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f2f0d-b91d-4183-aacf-ed32e9b612e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:19.595718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f2f0d-b91d-4183-aacf-ed32e9b612e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:19.595917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f2f0d-b91d-4183-aacf-ed32e9b612e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:19.597185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f2f0d-b91d-4183-aacf-ed32e9b612e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:19.604993 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f2f0d-b91d-4183-aacf-ed32e9b612e5 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:19.678486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 734/1436] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:19 2022] POST /identity/v3/projects => generated 387 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:19.681906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef245965-4bf8-467a-abbb-f194bc13f18f None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:19.702821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef245965-4bf8-467a-abbb-f194bc13f18f None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:19.925411 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e622f-e550-47ea-87a8-c4bff55c4d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:19.927013 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e622f-e550-47ea-87a8-c4bff55c4d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:19.927278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e622f-e550-47ea-87a8-c4bff55c4d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:19.932044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e622f-e550-47ea-87a8-c4bff55c4d0d admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:19.932956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e622f-e550-47ea-87a8-c4bff55c4d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:19.934915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e622f-e550-47ea-87a8-c4bff55c4d0d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:20.006860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 703/1437] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:19 2022] POST /identity/v3/users => generated 300 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:20.011401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f2f0d-b91d-4183-aacf-ed32e9b612e5 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:20.033947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f2f0d-b91d-4183-aacf-ed32e9b612e5 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:20.051567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9d817-5425-4d80-8306-443e8a59fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:20.055944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9d817-5425-4d80-8306-443e8a59fbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:20.057375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9d817-5425-4d80-8306-443e8a59fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:20.057667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9d817-5425-4d80-8306-443e8a59fbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:20.058776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9d817-5425-4d80-8306-443e8a59fbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:20.063596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9d817-5425-4d80-8306-443e8a59fbc2 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:20.112538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 735/1438] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:20.114045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e622f-e550-47ea-87a8-c4bff55c4d0d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:20.133197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e622f-e550-47ea-87a8-c4bff55c4d0d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:20.154155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bab3b-9c4f-4d9a-9683-c1de8e0c9704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:20.164937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bab3b-9c4f-4d9a-9683-c1de8e0c9704 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:20.165747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bab3b-9c4f-4d9a-9683-c1de8e0c9704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:20.166378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bab3b-9c4f-4d9a-9683-c1de8e0c9704 admin admin] PATH_INFO: `/v3/projects/dbb785ac634049fab5048d6c2290c2f4/users/79ffbc81cff341b183249065bc8bdf86/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:20.239467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bab3b-9c4f-4d9a-9683-c1de8e0c9704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb785ac634049fab5048d6c2290c2f4, user_id=79ffbc81cff341b183249065bc8bdf86, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:20.242971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bab3b-9c4f-4d9a-9683-c1de8e0c9704 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:20.294128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 704/1439] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:20 2022] PUT /identity/v3/projects/dbb785ac634049fab5048d6c2290c2f4/users/79ffbc81cff341b183249065bc8bdf86/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:20.298364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9d817-5425-4d80-8306-443e8a59fbc2 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:20.310599 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9d817-5425-4d80-8306-443e8a59fbc2 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:20.390517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455c267-3a10-4634-a73b-615578ac6e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:20.391150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455c267-3a10-4634-a73b-615578ac6e7f admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:20.391587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455c267-3a10-4634-a73b-615578ac6e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:20.391871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455c267-3a10-4634-a73b-615578ac6e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:20.392634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455c267-3a10-4634-a73b-615578ac6e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:20.394333 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455c267-3a10-4634-a73b-615578ac6e7f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:20.406879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:20.422476 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 736/1440] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:55:20.423951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bab3b-9c4f-4d9a-9683-c1de8e0c9704 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:20.452077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bab3b-9c4f-4d9a-9683-c1de8e0c9704 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:20.472649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce60481-9404-439c-99b9-d764008fe954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:20.473491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce60481-9404-439c-99b9-d764008fe954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:20.473772 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce60481-9404-439c-99b9-d764008fe954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:20.476364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce60481-9404-439c-99b9-d764008fe954 admin admin] PATH_INFO: `/v3/projects/dbb785ac634049fab5048d6c2290c2f4/users/79ffbc81cff341b183249065bc8bdf86/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:20.483701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce60481-9404-439c-99b9-d764008fe954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb785ac634049fab5048d6c2290c2f4, user_id=79ffbc81cff341b183249065bc8bdf86, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:20.490827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce60481-9404-439c-99b9-d764008fe954 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:20.544924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 705/1441] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:20 2022] PUT /identity/v3/projects/dbb785ac634049fab5048d6c2290c2f4/users/79ffbc81cff341b183249065bc8bdf86/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:20.553009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12170464-04c6-4a84-b2ba-aa22232311df None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:20.555047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12170464-04c6-4a84-b2ba-aa22232311df None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:20.555626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12170464-04c6-4a84-b2ba-aa22232311df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:20.669619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-12170464-04c6-4a84-b2ba-aa22232311df None None] MFA Rules not processed for user `79ffbc81cff341b183249065bc8bdf86`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:20.674520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-12170464-04c6-4a84-b2ba-aa22232311df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:20.922116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 737/1442] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:20.938540 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce60481-9404-439c-99b9-d764008fe954 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:21.068944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce60481-9404-439c-99b9-d764008fe954 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:21.082057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4a7df-dd06-4567-a86b-4c1bd0587217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:21.087287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4a7df-dd06-4567-a86b-4c1bd0587217 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:21.088864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4a7df-dd06-4567-a86b-4c1bd0587217 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:21.090053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4a7df-dd06-4567-a86b-4c1bd0587217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:21.125607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4a7df-dd06-4567-a86b-4c1bd0587217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:21.129298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4a7df-dd06-4567-a86b-4c1bd0587217 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:21.145719 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:21.160111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12170464-04c6-4a84-b2ba-aa22232311df None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:21.164710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 706/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:55:21.270701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4a7df-dd06-4567-a86b-4c1bd0587217 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:21.338824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12170464-04c6-4a84-b2ba-aa22232311df None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:21.365163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:21.366000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:21.366933 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:21.368788 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] PATH_INFO: `/v3/users/acefc47dc32044e9a52f7fbedd1293c8` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:21.371346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acefc47dc32044e9a52f7fbedd1293c8)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:21.373782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:21.497624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4a7df-dd06-4567-a86b-4c1bd0587217 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:21.513156 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:21.514037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:21.514590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:21.515010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] PATH_INFO: `/v3/users/f9e2b4a852f6446a8055a56fe658521c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:21.518437 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9e2b4a852f6446a8055a56fe658521c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:21.521002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:21.532669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acefc47dc32044e9a52f7fbedd1293c8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:21.545438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acefc47dc32044e9a52f7fbedd1293c8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:21.586767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acefc47dc32044e9a52f7fbedd1293c8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:21.587478 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acefc47dc32044e9a52f7fbedd1293c8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:21.637166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3d743e9-2928-41ae-8aea-2c3515b841c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acefc47dc32044e9a52f7fbedd1293c8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:21.720800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9e2b4a852f6446a8055a56fe658521c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:21.721223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 738/1444] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:21 2022] DELETE /identity/v3/users/acefc47dc32044e9a52f7fbedd1293c8 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:21.722620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0e5f7-0f32-47a9-8cef-336db1145d35 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:21.722900 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0e5f7-0f32-47a9-8cef-336db1145d35 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:21.723172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0e5f7-0f32-47a9-8cef-336db1145d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:21.727807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9e2b4a852f6446a8055a56fe658521c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:21.752093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9e2b4a852f6446a8055a56fe658521c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:21.752900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9e2b4a852f6446a8055a56fe658521c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:21.764190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:21.771986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e93c906e-22c9-4e75-82cf-56266d3db6ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9e2b4a852f6446a8055a56fe658521c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:21.808530 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-28a0e5f7-0f32-47a9-8cef-336db1145d35 None None] MFA Rules not processed for user `b9187955c3cf4b77ba86e67a6f378e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:21.810073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-28a0e5f7-0f32-47a9-8cef-336db1145d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:21.837348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 707/1445] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:21 2022] DELETE /identity/v3/users/f9e2b4a852f6446a8055a56fe658521c => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:21.838940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31017a93-6a89-433f-a932-6421561ad46c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:21.839225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31017a93-6a89-433f-a932-6421561ad46c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:21.839503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31017a93-6a89-433f-a932-6421561ad46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:21.908042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-31017a93-6a89-433f-a932-6421561ad46c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:21.920485 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-31017a93-6a89-433f-a932-6421561ad46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:21.965517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 739/1446] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:21.965517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe97d04-f37c-44bc-b963-af3e48eb85ac None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:21.965517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe97d04-f37c-44bc-b963-af3e48eb85ac None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:21.965517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe97d04-f37c-44bc-b963-af3e48eb85ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:22.033966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4fe97d04-f37c-44bc-b963-af3e48eb85ac None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:22.035005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4fe97d04-f37c-44bc-b963-af3e48eb85ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:22.086875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 708/1447] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:22.098542 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31017a93-6a89-433f-a932-6421561ad46c None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:22.100653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-31017a93-6a89-433f-a932-6421561ad46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:22.169013 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 740/1448] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:22.180810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:22.182655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe97d04-f37c-44bc-b963-af3e48eb85ac None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:22.186258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4fe97d04-f37c-44bc-b963-af3e48eb85ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:22.255652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31017a93-6a89-433f-a932-6421561ad46c None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:22.260508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd84279-d1bd-441b-b615-4c8b6eb2434d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:22.261604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd84279-d1bd-441b-b615-4c8b6eb2434d service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:22.262102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd84279-d1bd-441b-b615-4c8b6eb2434d service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:22.262624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd84279-d1bd-441b-b615-4c8b6eb2434d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:22.280931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd84279-d1bd-441b-b615-4c8b6eb2434d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:22.287053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd84279-d1bd-441b-b615-4c8b6eb2434d service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:22.315715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 709/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:22.328858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe97d04-f37c-44bc-b963-af3e48eb85ac None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:22.335842 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1a4f-9dac-4a23-80c6-2d6226303aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:22.337942 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1a4f-9dac-4a23-80c6-2d6226303aba service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:22.338270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1a4f-9dac-4a23-80c6-2d6226303aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:22.338639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1a4f-9dac-4a23-80c6-2d6226303aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:22.353431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf1a4f-9dac-4a23-80c6-2d6226303aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:22.355324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf1a4f-9dac-4a23-80c6-2d6226303aba service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:22.371625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 741/1450] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:23.393033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd84279-d1bd-441b-b615-4c8b6eb2434d service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:23.395029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-afd84279-d1bd-441b-b615-4c8b6eb2434d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:23.435696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1a4f-9dac-4a23-80c6-2d6226303aba service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:23.437296 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-abaf1a4f-9dac-4a23-80c6-2d6226303aba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:23.447743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd84279-d1bd-441b-b615-4c8b6eb2434d service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:23.495345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1a4f-9dac-4a23-80c6-2d6226303aba service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:23.513031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:23.513697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:23.513977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:23.514256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] PATH_INFO: `/v3/projects/17a23a40169b43db87861e5a3b0afa3f` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:23.517002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a23a40169b43db87861e5a3b0afa3f)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:23.527724 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:23.539759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:23.540447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:23.540732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:23.541011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] PATH_INFO: `/v3/projects/069be3feed4a45bbb848710c19d28052` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:23.552295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=069be3feed4a45bbb848710c19d28052)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:23.554163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:23.682772 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a23a40169b43db87861e5a3b0afa3f', 'request_id': 'req-6078526c-5acd-48aa-ba01-9a563296c93b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:23.699288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:23.700496 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a23a40169b43db87861e5a3b0afa3f', 'request_id': 'req-6078526c-5acd-48aa-ba01-9a563296c93b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:23.703233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069be3feed4a45bbb848710c19d28052', 'request_id': 'req-3a8f62de-0869-486a-86cc-4cb15be77792'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:23.714608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069be3feed4a45bbb848710c19d28052', 'request_id': 'req-3a8f62de-0869-486a-86cc-4cb15be77792'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:23.720629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a23a40169b43db87861e5a3b0afa3f', 'request_id': 'req-6078526c-5acd-48aa-ba01-9a563296c93b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:23.737473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] The token cache is being invalidate because project 17a23a40169b43db87861e5a3b0afa3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:23.737940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:23.739192 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:23.740900 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069be3feed4a45bbb848710c19d28052', 'request_id': 'req-3a8f62de-0869-486a-86cc-4cb15be77792'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:23.748927 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 710/1451] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:23 2022] DELETE /identity/v3/projects/17a23a40169b43db87861e5a3b0afa3f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:23.754270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:23.755742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] The token cache is being invalidate because project 069be3feed4a45bbb848710c19d28052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:23.756395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:23.757155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:23.758906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:23.762420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 742/1452] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:23 2022] DELETE /identity/v3/projects/069be3feed4a45bbb848710c19d28052 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:23.763291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:23.769867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:23.894863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f62de-0869-486a-86cc-4cb15be77792 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:23.906156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:23.906808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:23.907121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:23.907607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] PATH_INFO: `/v3/users/707685cd46974201ba709615ce483d1c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:23.920336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=707685cd46974201ba709615ce483d1c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:23.922620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:23.954661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078526c-5acd-48aa-ba01-9a563296c93b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:23.994977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:23.995841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:23.997064 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:23.997494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] PATH_INFO: `/v3/users/79ffbc81cff341b183249065bc8bdf86` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:24.004118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ffbc81cff341b183249065bc8bdf86)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:24.007608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:24.056138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707685cd46974201ba709615ce483d1c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:24.056768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707685cd46974201ba709615ce483d1c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:24.069475 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707685cd46974201ba709615ce483d1c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:24.071341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707685cd46974201ba709615ce483d1c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:24.090871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707685cd46974201ba709615ce483d1c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:24.121817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 743/1453] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:23 2022] DELETE /identity/v3/users/707685cd46974201ba709615ce483d1c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:24.177436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ffbc81cff341b183249065bc8bdf86'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:24.180593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ffbc81cff341b183249065bc8bdf86'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:24.184331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:24.202586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ffbc81cff341b183249065bc8bdf86'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:24.209687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ffbc81cff341b183249065bc8bdf86'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:24.249858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ffbc81cff341b183249065bc8bdf86'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:24.299651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 711/1454] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:23 2022] DELETE /identity/v3/users/79ffbc81cff341b183249065bc8bdf86 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:24.863638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:24.865826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:24.953981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:24.955944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:24.970403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:24.978230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ef8a7-bda1-4eca-b8b6-c5d14d77da7f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.056093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5063f52-2441-40a3-b6d4-a05425935cd7 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.058144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.058784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.059061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.059330 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] PATH_INFO: `/v3/projects/35634575726d46d3a6736b3f5b25f50a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.060501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35634575726d46d3a6736b3f5b25f50a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.062426 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.067464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.068262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.068544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.068845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] PATH_INFO: `/v3/projects/dbb785ac634049fab5048d6c2290c2f4` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.069990 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb785ac634049fab5048d6c2290c2f4)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.072511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.122318 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35634575726d46d3a6736b3f5b25f50a', 'request_id': 'req-49eac410-98b5-459a-b742-55e838aed1c5'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:25.128899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35634575726d46d3a6736b3f5b25f50a', 'request_id': 'req-49eac410-98b5-459a-b742-55e838aed1c5'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:25.132385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb785ac634049fab5048d6c2290c2f4', 'request_id': 'req-5053b07a-d991-4894-8689-0dc86d13cea1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:25.141553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35634575726d46d3a6736b3f5b25f50a', 'request_id': 'req-49eac410-98b5-459a-b742-55e838aed1c5'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:25.142133 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb785ac634049fab5048d6c2290c2f4', 'request_id': 'req-5053b07a-d991-4894-8689-0dc86d13cea1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:25.152584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb785ac634049fab5048d6c2290c2f4', 'request_id': 'req-5053b07a-d991-4894-8689-0dc86d13cea1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:25.159803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] The token cache is being invalidate because project 35634575726d46d3a6736b3f5b25f50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:25.160264 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:25.161019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-49eac410-98b5-459a-b742-55e838aed1c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:25.164036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 744/1455] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:24 2022] DELETE /identity/v3/projects/35634575726d46d3a6736b3f5b25f50a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:25.168708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] The token cache is being invalidate because project dbb785ac634049fab5048d6c2290c2f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:25.169208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:25.170412 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5053b07a-d991-4894-8689-0dc86d13cea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:25.172851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 712/1456] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:24 2022] DELETE /identity/v3/projects/dbb785ac634049fab5048d6c2290c2f4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:25.178302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca126-3942-4e6b-b4dd-42d633200098 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.178585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca126-3942-4e6b-b4dd-42d633200098 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.178856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca126-3942-4e6b-b4dd-42d633200098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.180427 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ad157-7a4e-4799-883a-6a775c0295df None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.180708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ad157-7a4e-4799-883a-6a775c0295df None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.181027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ad157-7a4e-4799-883a-6a775c0295df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.214576 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-070ad157-7a4e-4799-883a-6a775c0295df None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:25.214768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-3b5ca126-3942-4e6b-b4dd-42d633200098 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:25.215791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-070ad157-7a4e-4799-883a-6a775c0295df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:25.216163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3b5ca126-3942-4e6b-b4dd-42d633200098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:25.271558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:25.294170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 745/1457] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:25.295130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 713/1458] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:25.307383 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce0077-b2d0-4968-9f1f-d8dbb25226ae None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.307662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce0077-b2d0-4968-9f1f-d8dbb25226ae None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.307961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce0077-b2d0-4968-9f1f-d8dbb25226ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.310025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c0f93-d534-4c08-9b4d-e52aed8005a1 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.310558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c0f93-d534-4c08-9b4d-e52aed8005a1 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.311052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c0f93-d534-4c08-9b4d-e52aed8005a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.329637 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-82ce0077-b2d0-4968-9f1f-d8dbb25226ae None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:25.330992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-82ce0077-b2d0-4968-9f1f-d8dbb25226ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:25.335416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-039c0f93-d534-4c08-9b4d-e52aed8005a1 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:25.337285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-039c0f93-d534-4c08-9b4d-e52aed8005a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:25.342722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 714/1459] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:25.350633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce0077-b2d0-4968-9f1f-d8dbb25226ae None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.353235 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 746/1460] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:25.357602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c0f93-d534-4c08-9b4d-e52aed8005a1 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.363537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce0077-b2d0-4968-9f1f-d8dbb25226ae None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.367809 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c0f93-d534-4c08-9b4d-e52aed8005a1 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.374378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42552b27-cbf0-4ac3-b31e-9e9dfcff626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.374378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42552b27-cbf0-4ac3-b31e-9e9dfcff626c admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.374378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42552b27-cbf0-4ac3-b31e-9e9dfcff626c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.374378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42552b27-cbf0-4ac3-b31e-9e9dfcff626c admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.374792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42552b27-cbf0-4ac3-b31e-9e9dfcff626c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:25.374792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42552b27-cbf0-4ac3-b31e-9e9dfcff626c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.374941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bde42-3060-4bcc-997c-6bb81fa09236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.374941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bde42-3060-4bcc-997c-6bb81fa09236 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.374941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bde42-3060-4bcc-997c-6bb81fa09236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.374941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bde42-3060-4bcc-997c-6bb81fa09236 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.375241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bde42-3060-4bcc-997c-6bb81fa09236 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:25.375510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bde42-3060-4bcc-997c-6bb81fa09236 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.375677 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42552b27-cbf0-4ac3-b31e-9e9dfcff626c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.377462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bde42-3060-4bcc-997c-6bb81fa09236 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.392130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 747/1461] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:55:25.394279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bde42-3060-4bcc-997c-6bb81fa09236 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.403580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 715/1462] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:25.405934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42552b27-cbf0-4ac3-b31e-9e9dfcff626c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.407740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bde42-3060-4bcc-997c-6bb81fa09236 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.413331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884dfb2-ee70-4821-b6c8-395c2d0795f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.416081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884dfb2-ee70-4821-b6c8-395c2d0795f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.416594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884dfb2-ee70-4821-b6c8-395c2d0795f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.417180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884dfb2-ee70-4821-b6c8-395c2d0795f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.418024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884dfb2-ee70-4821-b6c8-395c2d0795f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.419250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42552b27-cbf0-4ac3-b31e-9e9dfcff626c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.420568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884dfb2-ee70-4821-b6c8-395c2d0795f9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.424122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763b521-d8b9-4971-a05d-ada529cf8e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.424723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763b521-d8b9-4971-a05d-ada529cf8e26 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.425003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763b521-d8b9-4971-a05d-ada529cf8e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.425276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763b521-d8b9-4971-a05d-ada529cf8e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.425905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763b521-d8b9-4971-a05d-ada529cf8e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.428117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763b521-d8b9-4971-a05d-ada529cf8e26 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.435284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:25.458545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 748/1463] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/projects => generated 409 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:55:25.459385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884dfb2-ee70-4821-b6c8-395c2d0795f9 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.465448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 716/1464] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:25.466381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763b521-d8b9-4971-a05d-ada529cf8e26 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.542352 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763b521-d8b9-4971-a05d-ada529cf8e26 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.547101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e739502-7016-4f54-9edd-e2397bf1db13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.547699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e739502-7016-4f54-9edd-e2397bf1db13 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.548052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e739502-7016-4f54-9edd-e2397bf1db13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.548288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e739502-7016-4f54-9edd-e2397bf1db13 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.549547 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e739502-7016-4f54-9edd-e2397bf1db13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.551052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e739502-7016-4f54-9edd-e2397bf1db13 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.553613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884dfb2-ee70-4821-b6c8-395c2d0795f9 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.562313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4af1d-c0ed-49b4-bf89-d4e956e45102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.563457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4af1d-c0ed-49b4-bf89-d4e956e45102 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.563735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4af1d-c0ed-49b4-bf89-d4e956e45102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.564046 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4af1d-c0ed-49b4-bf89-d4e956e45102 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.564747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4af1d-c0ed-49b4-bf89-d4e956e45102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.566740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4af1d-c0ed-49b4-bf89-d4e956e45102 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.574143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 717/1465] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:55:25.580112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e739502-7016-4f54-9edd-e2397bf1db13 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.596522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e739502-7016-4f54-9edd-e2397bf1db13 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.604904 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 749/1466] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/users => generated 305 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:25.607309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff23280-7d29-4168-89ca-ddd865679897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.607937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff23280-7d29-4168-89ca-ddd865679897 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.608331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff23280-7d29-4168-89ca-ddd865679897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.608484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff23280-7d29-4168-89ca-ddd865679897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.609144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff23280-7d29-4168-89ca-ddd865679897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.609387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4af1d-c0ed-49b4-bf89-d4e956e45102 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.610846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff23280-7d29-4168-89ca-ddd865679897 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.620484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4af1d-c0ed-49b4-bf89-d4e956e45102 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.622353 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 718/1467] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:25.625378 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee123d-66d4-44b5-9297-a26438527ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.625988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee123d-66d4-44b5-9297-a26438527ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.626136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff23280-7d29-4168-89ca-ddd865679897 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.626265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee123d-66d4-44b5-9297-a26438527ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.626538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee123d-66d4-44b5-9297-a26438527ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.627176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee123d-66d4-44b5-9297-a26438527ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.628813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee123d-66d4-44b5-9297-a26438527ce6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.635730 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff23280-7d29-4168-89ca-ddd865679897 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.642366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 750/1468] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:25.642505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f02d43-c0a4-4c0f-84c4-6981594ee47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.642505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f02d43-c0a4-4c0f-84c4-6981594ee47f admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.642505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f02d43-c0a4-4c0f-84c4-6981594ee47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.642505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f02d43-c0a4-4c0f-84c4-6981594ee47f admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.642774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f02d43-c0a4-4c0f-84c4-6981594ee47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.643432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee123d-66d4-44b5-9297-a26438527ce6 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.644077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f02d43-c0a4-4c0f-84c4-6981594ee47f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.653713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee123d-66d4-44b5-9297-a26438527ce6 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.656522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 719/1469] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:55:25.659092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadb01c-ddf5-4d62-9a14-72b681f466bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.659690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadb01c-ddf5-4d62-9a14-72b681f466bb admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.660001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f02d43-c0a4-4c0f-84c4-6981594ee47f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.660114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadb01c-ddf5-4d62-9a14-72b681f466bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.660387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadb01c-ddf5-4d62-9a14-72b681f466bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.661005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadb01c-ddf5-4d62-9a14-72b681f466bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.662801 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadb01c-ddf5-4d62-9a14-72b681f466bb admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.670107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f02d43-c0a4-4c0f-84c4-6981594ee47f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.674464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 751/1470] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:25.675166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04407e-39d8-43e4-9baa-20d30e0b7a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.675810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04407e-39d8-43e4-9baa-20d30e0b7a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.676117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04407e-39d8-43e4-9baa-20d30e0b7a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.676387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04407e-39d8-43e4-9baa-20d30e0b7a2d admin admin] PATH_INFO: `/v3/projects/ee40f2f0dd7e46eaa91cee4a7e26496d/users/b89f68d663164421b4c15e31d0b8623b/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.678358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadb01c-ddf5-4d62-9a14-72b681f466bb admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.688436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadb01c-ddf5-4d62-9a14-72b681f466bb admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.691118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04407e-39d8-43e4-9baa-20d30e0b7a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee40f2f0dd7e46eaa91cee4a7e26496d, user_id=b89f68d663164421b4c15e31d0b8623b, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.693466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc025a93-2ca4-41c2-95ef-723c5a5e9be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.694034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04407e-39d8-43e4-9baa-20d30e0b7a2d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.694213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc025a93-2ca4-41c2-95ef-723c5a5e9be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.694491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc025a93-2ca4-41c2-95ef-723c5a5e9be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.694759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc025a93-2ca4-41c2-95ef-723c5a5e9be3 admin admin] PATH_INFO: `/v3/projects/49aafe383fa141e796cff32be0842b82/users/db60633510b041fb9008105119fee5ba/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.708122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 720/1471] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:25 2022] PUT /identity/v3/projects/ee40f2f0dd7e46eaa91cee4a7e26496d/users/b89f68d663164421b4c15e31d0b8623b/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:25.710936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc025a93-2ca4-41c2-95ef-723c5a5e9be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49aafe383fa141e796cff32be0842b82, user_id=db60633510b041fb9008105119fee5ba, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.714270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc025a93-2ca4-41c2-95ef-723c5a5e9be3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.719385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6073a-2d8c-4bad-b08e-519cce178ea6 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.719702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6073a-2d8c-4bad-b08e-519cce178ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.720016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6073a-2d8c-4bad-b08e-519cce178ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.727946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 752/1472] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:25 2022] PUT /identity/v3/projects/49aafe383fa141e796cff32be0842b82/users/db60633510b041fb9008105119fee5ba/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:25.738263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c68a-054b-4f1f-9e71-1556a85dd1f4 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.738553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c68a-054b-4f1f-9e71-1556a85dd1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.738828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c68a-054b-4f1f-9e71-1556a85dd1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.754734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-26a6073a-2d8c-4bad-b08e-519cce178ea6 None None] MFA Rules not processed for user `b89f68d663164421b4c15e31d0b8623b`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:25.755962 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-26a6073a-2d8c-4bad-b08e-519cce178ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:25.768882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8c34c68a-054b-4f1f-9e71-1556a85dd1f4 None None] MFA Rules not processed for user `db60633510b041fb9008105119fee5ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:25.770337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8c34c68a-054b-4f1f-9e71-1556a85dd1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:25.816646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:25.852210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 721/1473] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:25.860988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6073a-2d8c-4bad-b08e-519cce178ea6 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.861452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 753/1474] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:25.865869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34c68a-054b-4f1f-9e71-1556a85dd1f4 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:25.903209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:25.938839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6073a-2d8c-4bad-b08e-519cce178ea6 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.949348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d27a8-c4fd-4ed4-91b2-0439c4cd1dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.950801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d27a8-c4fd-4ed4-91b2-0439c4cd1dfc admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.951228 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d27a8-c4fd-4ed4-91b2-0439c4cd1dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.951726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d27a8-c4fd-4ed4-91b2-0439c4cd1dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.952966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d27a8-c4fd-4ed4-91b2-0439c4cd1dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.953767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34c68a-054b-4f1f-9e71-1556a85dd1f4 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:25.958397 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b17c9c-418b-4b52-81be-efdeccecf547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:25.959082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b17c9c-418b-4b52-81be-efdeccecf547 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:25.959365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b17c9c-418b-4b52-81be-efdeccecf547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:25.959625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b17c9c-418b-4b52-81be-efdeccecf547 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:25.960378 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b17c9c-418b-4b52-81be-efdeccecf547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:25.962715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b17c9c-418b-4b52-81be-efdeccecf547 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:25.962909 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d27a8-c4fd-4ed4-91b2-0439c4cd1dfc admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.010464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 722/1475] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:55:26.015514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d27a8-c4fd-4ed4-91b2-0439c4cd1dfc admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.019419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 754/1476] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:25 2022] POST /identity/v3/projects => generated 409 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:55:26.026798 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b17c9c-418b-4b52-81be-efdeccecf547 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.123723 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b17c9c-418b-4b52-81be-efdeccecf547 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.129330 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe02e57-fcef-4d60-bfc4-f4e4efd1fcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.130198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe02e57-fcef-4d60-bfc4-f4e4efd1fcae admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.130198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe02e57-fcef-4d60-bfc4-f4e4efd1fcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.130654 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe02e57-fcef-4d60-bfc4-f4e4efd1fcae admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.131670 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe02e57-fcef-4d60-bfc4-f4e4efd1fcae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.134712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe02e57-fcef-4d60-bfc4-f4e4efd1fcae admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.140245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d27a8-c4fd-4ed4-91b2-0439c4cd1dfc admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.150360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81690f13-f798-4519-acd9-230cf4ce8b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.151561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81690f13-f798-4519-acd9-230cf4ce8b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.152038 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81690f13-f798-4519-acd9-230cf4ce8b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.152466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81690f13-f798-4519-acd9-230cf4ce8b5e admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.153269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81690f13-f798-4519-acd9-230cf4ce8b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.155493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81690f13-f798-4519-acd9-230cf4ce8b5e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.170893 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 755/1477] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:26 2022] POST /identity/v3/users => generated 311 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:26.172824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe02e57-fcef-4d60-bfc4-f4e4efd1fcae admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.198522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 723/1478] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:26 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:26.199269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81690f13-f798-4519-acd9-230cf4ce8b5e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.199685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe02e57-fcef-4d60-bfc4-f4e4efd1fcae admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.208209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394191ed-d6cf-49d8-9c5d-774bb9f36410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.208977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394191ed-d6cf-49d8-9c5d-774bb9f36410 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.209374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394191ed-d6cf-49d8-9c5d-774bb9f36410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.209712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394191ed-d6cf-49d8-9c5d-774bb9f36410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.210492 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394191ed-d6cf-49d8-9c5d-774bb9f36410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.213917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394191ed-d6cf-49d8-9c5d-774bb9f36410 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.216440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81690f13-f798-4519-acd9-230cf4ce8b5e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.226051 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f24f9-04d8-4fb4-b9ce-ddf48ace5b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.227147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f24f9-04d8-4fb4-b9ce-ddf48ace5b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.227740 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f24f9-04d8-4fb4-b9ce-ddf48ace5b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.228228 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f24f9-04d8-4fb4-b9ce-ddf48ace5b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.229757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f24f9-04d8-4fb4-b9ce-ddf48ace5b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.232815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f24f9-04d8-4fb4-b9ce-ddf48ace5b18 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.234317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 756/1479] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:26.241226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394191ed-d6cf-49d8-9c5d-774bb9f36410 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.247863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 724/1480] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:55:26.253337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f24f9-04d8-4fb4-b9ce-ddf48ace5b18 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.255496 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394191ed-d6cf-49d8-9c5d-774bb9f36410 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.262291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bf476-986c-4e5a-ac78-b0173638688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.262982 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bf476-986c-4e5a-ac78-b0173638688e admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.263342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bf476-986c-4e5a-ac78-b0173638688e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.263613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bf476-986c-4e5a-ac78-b0173638688e admin admin] PATH_INFO: `/v3/projects/1500cdd38e944ea280fd55a9c30ab914/users/b3494c84f3f842aa8149cd3f104d6e02/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.266713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f24f9-04d8-4fb4-b9ce-ddf48ace5b18 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.271907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0472d-800f-41c9-81b2-837a19189d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.273041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0472d-800f-41c9-81b2-837a19189d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.273560 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0472d-800f-41c9-81b2-837a19189d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.274182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0472d-800f-41c9-81b2-837a19189d18 admin admin] PATH_INFO: `/v3/projects/f671b65573b245559a27eca54dbe4dcb/users/6187de6906e54eed9419fa9203543f64/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.291589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0472d-800f-41c9-81b2-837a19189d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f671b65573b245559a27eca54dbe4dcb, user_id=6187de6906e54eed9419fa9203543f64, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.295053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bf476-986c-4e5a-ac78-b0173638688e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1500cdd38e944ea280fd55a9c30ab914, user_id=b3494c84f3f842aa8149cd3f104d6e02, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.295686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0472d-800f-41c9-81b2-837a19189d18 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.297794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bf476-986c-4e5a-ac78-b0173638688e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.325833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 757/1481] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:26 2022] PUT /identity/v3/projects/1500cdd38e944ea280fd55a9c30ab914/users/b3494c84f3f842aa8149cd3f104d6e02/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:26.331867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bf476-986c-4e5a-ac78-b0173638688e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.333000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 725/1482] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:26 2022] PUT /identity/v3/projects/f671b65573b245559a27eca54dbe4dcb/users/6187de6906e54eed9419fa9203543f64/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:26.337892 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0472d-800f-41c9-81b2-837a19189d18 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.386143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:26.442456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bf476-986c-4e5a-ac78-b0173638688e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.453257 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61014964-3067-4afa-8ea6-051779bfec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.453961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014964-3067-4afa-8ea6-051779bfec8c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.454315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014964-3067-4afa-8ea6-051779bfec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.454660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014964-3067-4afa-8ea6-051779bfec8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.455445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61014964-3067-4afa-8ea6-051779bfec8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.456409 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0472d-800f-41c9-81b2-837a19189d18 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.457334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61014964-3067-4afa-8ea6-051779bfec8c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.461997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51157b-75f4-4153-9e07-a61c340a2f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.462773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51157b-75f4-4153-9e07-a61c340a2f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.463257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51157b-75f4-4153-9e07-a61c340a2f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.463530 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51157b-75f4-4153-9e07-a61c340a2f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.464669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51157b-75f4-4153-9e07-a61c340a2f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.466303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51157b-75f4-4153-9e07-a61c340a2f71 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.470491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:26.474743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 758/1483] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:26.479176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61014964-3067-4afa-8ea6-051779bfec8c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.488569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 726/1484] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:26.490054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51157b-75f4-4153-9e07-a61c340a2f71 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.499139 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61014964-3067-4afa-8ea6-051779bfec8c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.502173 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51157b-75f4-4153-9e07-a61c340a2f71 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.506853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2f246-2288-43c6-96a8-3855ed334633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.507614 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2f246-2288-43c6-96a8-3855ed334633 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.508510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2f246-2288-43c6-96a8-3855ed334633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.508852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2f246-2288-43c6-96a8-3855ed334633 admin admin] PATH_INFO: `/v3/projects/1500cdd38e944ea280fd55a9c30ab914/users/b3494c84f3f842aa8149cd3f104d6e02/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.512688 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2f246-2288-43c6-96a8-3855ed334633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1500cdd38e944ea280fd55a9c30ab914, user_id=b3494c84f3f842aa8149cd3f104d6e02, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.516738 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2f246-2288-43c6-96a8-3855ed334633 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.526177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26793d1a-ae9e-4192-8cfd-2ca7d0adfdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.533686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26793d1a-ae9e-4192-8cfd-2ca7d0adfdb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.533975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26793d1a-ae9e-4192-8cfd-2ca7d0adfdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.534251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26793d1a-ae9e-4192-8cfd-2ca7d0adfdb2 admin admin] PATH_INFO: `/v3/projects/f671b65573b245559a27eca54dbe4dcb/users/6187de6906e54eed9419fa9203543f64/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.537288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 759/1485] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:26 2022] PUT /identity/v3/projects/1500cdd38e944ea280fd55a9c30ab914/users/b3494c84f3f842aa8149cd3f104d6e02/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:26.548932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26793d1a-ae9e-4192-8cfd-2ca7d0adfdb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f671b65573b245559a27eca54dbe4dcb, user_id=6187de6906e54eed9419fa9203543f64, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.549571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77ae5b-cefd-4cc5-8fad-7b8f7ed80238 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.549932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77ae5b-cefd-4cc5-8fad-7b8f7ed80238 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.550275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77ae5b-cefd-4cc5-8fad-7b8f7ed80238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.554712 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26793d1a-ae9e-4192-8cfd-2ca7d0adfdb2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.584390 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 727/1486] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:26 2022] PUT /identity/v3/projects/f671b65573b245559a27eca54dbe4dcb/users/6187de6906e54eed9419fa9203543f64/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:26.586937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d00254-5da0-4e34-8b6c-4b37fb05d62a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.587102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d00254-5da0-4e34-8b6c-4b37fb05d62a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.587377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d00254-5da0-4e34-8b6c-4b37fb05d62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.635347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9c77ae5b-cefd-4cc5-8fad-7b8f7ed80238 None None] MFA Rules not processed for user `b3494c84f3f842aa8149cd3f104d6e02`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:26.636909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9c77ae5b-cefd-4cc5-8fad-7b8f7ed80238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:26.646487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a0d00254-5da0-4e34-8b6c-4b37fb05d62a None None] MFA Rules not processed for user `6187de6906e54eed9419fa9203543f64`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:26.648260 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a0d00254-5da0-4e34-8b6c-4b37fb05d62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:26.795509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 760/1487] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:26 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:26.795696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 728/1488] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:26.803481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77ae5b-cefd-4cc5-8fad-7b8f7ed80238 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.803696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d00254-5da0-4e34-8b6c-4b37fb05d62a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:26.805730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9c77ae5b-cefd-4cc5-8fad-7b8f7ed80238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:26.805837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a0d00254-5da0-4e34-8b6c-4b37fb05d62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:26.875638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77ae5b-cefd-4cc5-8fad-7b8f7ed80238 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.880119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b328b1e-4240-44f9-8356-a8d4e7731ebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.881006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d00254-5da0-4e34-8b6c-4b37fb05d62a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:26.881140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b328b1e-4240-44f9-8356-a8d4e7731ebd service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.881140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b328b1e-4240-44f9-8356-a8d4e7731ebd service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.881294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b328b1e-4240-44f9-8356-a8d4e7731ebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.885268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1069619-d4f8-473c-9b3d-b3783736afac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:26.885948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1069619-d4f8-473c-9b3d-b3783736afac service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:26.886308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1069619-d4f8-473c-9b3d-b3783736afac service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:26.886639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1069619-d4f8-473c-9b3d-b3783736afac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:26.890702 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b328b1e-4240-44f9-8356-a8d4e7731ebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.892189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b328b1e-4240-44f9-8356-a8d4e7731ebd service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.902584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 761/1489] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:26 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:26.938451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:26.953858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1069619-d4f8-473c-9b3d-b3783736afac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:26.955372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1069619-d4f8-473c-9b3d-b3783736afac service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:26.966683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 729/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:55:27.107133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b328b1e-4240-44f9-8356-a8d4e7731ebd service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:27.118989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b328b1e-4240-44f9-8356-a8d4e7731ebd service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:27.136345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144c717-a9fc-40b9-bae0-1c6816ec588c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:27.137135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144c717-a9fc-40b9-bae0-1c6816ec588c service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:27.137589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144c717-a9fc-40b9-bae0-1c6816ec588c service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:27.137860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144c717-a9fc-40b9-bae0-1c6816ec588c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:27.200172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144c717-a9fc-40b9-bae0-1c6816ec588c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:27.201812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144c717-a9fc-40b9-bae0-1c6816ec588c service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:27.212807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 762/1491] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:27 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:28.558736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e01474-8080-441e-8e3d-fb970749ff90 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:28.559706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e01474-8080-441e-8e3d-fb970749ff90 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:28.560418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e01474-8080-441e-8e3d-fb970749ff90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:28.701708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-65e01474-8080-441e-8e3d-fb970749ff90 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:28.702943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-65e01474-8080-441e-8e3d-fb970749ff90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:28.907084 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 730/1492] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:55:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:29.183157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144c717-a9fc-40b9-bae0-1c6816ec588c service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:29.185104 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6144c717-a9fc-40b9-bae0-1c6816ec588c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:29.241662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:29.443091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144c717-a9fc-40b9-bae0-1c6816ec588c service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:29.469269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b694bc-64cb-41b7-8139-a15a59aafbd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:29.470596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b694bc-64cb-41b7-8139-a15a59aafbd5 service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:29.470883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b694bc-64cb-41b7-8139-a15a59aafbd5 service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:29.471162 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b694bc-64cb-41b7-8139-a15a59aafbd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:29.488338 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b694bc-64cb-41b7-8139-a15a59aafbd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:29.492068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b694bc-64cb-41b7-8139-a15a59aafbd5 service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:29.506683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 763/1493] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:55:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:55:31.065378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e01474-8080-441e-8e3d-fb970749ff90 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:31.065900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-65e01474-8080-441e-8e3d-fb970749ff90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:31.165606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e01474-8080-441e-8e3d-fb970749ff90 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:31.189615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:31.190959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:31.191958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:31.193067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] PATH_INFO: `/v3/users/7268d20c2b6f4f24b5c03bb57dcc6a0b` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:31.199902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7268d20c2b6f4f24b5c03bb57dcc6a0b)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:31.205972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:31.296855 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7268d20c2b6f4f24b5c03bb57dcc6a0b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:31.297684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7268d20c2b6f4f24b5c03bb57dcc6a0b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:31.305879 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7268d20c2b6f4f24b5c03bb57dcc6a0b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:31.306731 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7268d20c2b6f4f24b5c03bb57dcc6a0b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:31.312701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:31.324793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7268d20c2b6f4f24b5c03bb57dcc6a0b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:31.341106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 731/1494] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:31 2022] DELETE /identity/v3/users/7268d20c2b6f4f24b5c03bb57dcc6a0b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:31.345471 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b539fe4-4996-49ba-a69a-59171ae709ec None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:31.345979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b539fe4-4996-49ba-a69a-59171ae709ec None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:31.346206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b539fe4-4996-49ba-a69a-59171ae709ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:31.366492 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4b539fe4-4996-49ba-a69a-59171ae709ec None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:31.368663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4b539fe4-4996-49ba-a69a-59171ae709ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:31.432828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 764/1495] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:31.442596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:31.444107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:31.502442 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa42f7d-6119-4c74-b416-24f117ba6f97 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:31.506015 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ceded-9f82-42cc-9109-3669e060af88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:31.506618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ceded-9f82-42cc-9109-3669e060af88 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:31.506890 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ceded-9f82-42cc-9109-3669e060af88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:31.507158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ceded-9f82-42cc-9109-3669e060af88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:31.517072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ceded-9f82-42cc-9109-3669e060af88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:31.518514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ceded-9f82-42cc-9109-3669e060af88 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:31.528882 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 732/1496] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:31.826263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b539fe4-4996-49ba-a69a-59171ae709ec None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:31.829858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4b539fe4-4996-49ba-a69a-59171ae709ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:31.853408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b539fe4-4996-49ba-a69a-59171ae709ec None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:31.866447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:31.867861 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:31.868316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:31.868701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] PATH_INFO: `/v3/projects/e1e5f99ad67d454d904be9dc08c0411b` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:31.869972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e5f99ad67d454d904be9dc08c0411b)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:31.872005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:31.924459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e5f99ad67d454d904be9dc08c0411b', 'request_id': 'req-1204eec2-a161-44e3-978d-64a2e845879c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:31.931045 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e5f99ad67d454d904be9dc08c0411b', 'request_id': 'req-1204eec2-a161-44e3-978d-64a2e845879c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:31.938499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e5f99ad67d454d904be9dc08c0411b', 'request_id': 'req-1204eec2-a161-44e3-978d-64a2e845879c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:31.950187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] The token cache is being invalidate because project e1e5f99ad67d454d904be9dc08c0411b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:31.950491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:31.951152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:31.952681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 765/1497] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:31 2022] DELETE /identity/v3/projects/e1e5f99ad67d454d904be9dc08c0411b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:31.956683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ceded-9f82-42cc-9109-3669e060af88 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:31.966552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ceded-9f82-42cc-9109-3669e060af88 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:32.022483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:32.023118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:32.023918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:32.024225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] PATH_INFO: `/v3/users/73a24990b57c4e42892c1e590b3073e9` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:32.025381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a24990b57c4e42892c1e590b3073e9)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:32.027425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:32.074523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a24990b57c4e42892c1e590b3073e9'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:32.075220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a24990b57c4e42892c1e590b3073e9'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:32.082040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a24990b57c4e42892c1e590b3073e9'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:32.082667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a24990b57c4e42892c1e590b3073e9'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:32.094990 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-22dc0943-eb1f-4be4-a07c-648c5532068b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a24990b57c4e42892c1e590b3073e9'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:32.110174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 733/1498] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:31 2022] DELETE /identity/v3/users/73a24990b57c4e42892c1e590b3073e9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:32.392200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:32.393686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:32.409866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204eec2-a161-44e3-978d-64a2e845879c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:32.416116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:32.476194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:32.477563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:32.477836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:32.478103 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] PATH_INFO: `/v3/projects/3033d08dbe484d0a95dcaa46525cedf8` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:32.479213 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3033d08dbe484d0a95dcaa46525cedf8)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:32.481154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:32.527413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3033d08dbe484d0a95dcaa46525cedf8', 'request_id': 'req-f2f51b6b-712c-4320-a752-bb244b4394fb'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:32.533014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3033d08dbe484d0a95dcaa46525cedf8', 'request_id': 'req-f2f51b6b-712c-4320-a752-bb244b4394fb'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:32.540437 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3033d08dbe484d0a95dcaa46525cedf8', 'request_id': 'req-f2f51b6b-712c-4320-a752-bb244b4394fb'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:32.551192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] The token cache is being invalidate because project 3033d08dbe484d0a95dcaa46525cedf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:32.551468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:32.552115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f2f51b6b-712c-4320-a752-bb244b4394fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:32.553943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 766/1499] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:32 2022] DELETE /identity/v3/projects/3033d08dbe484d0a95dcaa46525cedf8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:32.558511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f8741-1f7c-46d0-b35d-b5e7dfe3b723 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:32.558798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f8741-1f7c-46d0-b35d-b5e7dfe3b723 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:32.559078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f8741-1f7c-46d0-b35d-b5e7dfe3b723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:32.567441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:32.582699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f60f8741-1f7c-46d0-b35d-b5e7dfe3b723 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:32.583941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f60f8741-1f7c-46d0-b35d-b5e7dfe3b723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:32.644231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 734/1500] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:32.654272 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82453f4-2689-46b0-8c02-320a45ab1d8d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:32.654551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82453f4-2689-46b0-8c02-320a45ab1d8d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:32.654815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82453f4-2689-46b0-8c02-320a45ab1d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:32.674594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f82453f4-2689-46b0-8c02-320a45ab1d8d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:32.675778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f82453f4-2689-46b0-8c02-320a45ab1d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:32.684129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 767/1501] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:32.688349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f8741-1f7c-46d0-b35d-b5e7dfe3b723 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:32.698541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f8741-1f7c-46d0-b35d-b5e7dfe3b723 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:32.703035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732731b8-a9b1-4c6d-85f5-f61ee0d3d209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:32.703666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732731b8-a9b1-4c6d-85f5-f61ee0d3d209 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:32.703966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732731b8-a9b1-4c6d-85f5-f61ee0d3d209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:32.704238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732731b8-a9b1-4c6d-85f5-f61ee0d3d209 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:32.704875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732731b8-a9b1-4c6d-85f5-f61ee0d3d209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:32.705173 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732731b8-a9b1-4c6d-85f5-f61ee0d3d209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:32.706805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732731b8-a9b1-4c6d-85f5-f61ee0d3d209 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:32.718793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 735/1502] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:32.722884 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82453f4-2689-46b0-8c02-320a45ab1d8d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:32.732285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82453f4-2689-46b0-8c02-320a45ab1d8d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:32.736789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dce197-2557-43b2-8011-015f2f6d4e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:32.737422 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dce197-2557-43b2-8011-015f2f6d4e92 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:32.737678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dce197-2557-43b2-8011-015f2f6d4e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:32.738045 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dce197-2557-43b2-8011-015f2f6d4e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:32.738700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dce197-2557-43b2-8011-015f2f6d4e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:32.740665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dce197-2557-43b2-8011-015f2f6d4e92 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:32.782809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732731b8-a9b1-4c6d-85f5-f61ee0d3d209 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:32.785850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 768/1503] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:32 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:32.798633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732731b8-a9b1-4c6d-85f5-f61ee0d3d209 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:32.893924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df7b61-c070-4001-a0b4-a123f20d7637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:32.894581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df7b61-c070-4001-a0b4-a123f20d7637 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:32.894853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df7b61-c070-4001-a0b4-a123f20d7637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:32.898538 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df7b61-c070-4001-a0b4-a123f20d7637 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:32.900087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df7b61-c070-4001-a0b4-a123f20d7637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:32.902065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df7b61-c070-4001-a0b4-a123f20d7637 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:32.930390 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 736/1504] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:32 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:32.934741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dce197-2557-43b2-8011-015f2f6d4e92 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:32.954547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dce197-2557-43b2-8011-015f2f6d4e92 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:32.970812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b77f-f3e6-438d-8c29-8e4cb6668527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:32.971433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b77f-f3e6-438d-8c29-8e4cb6668527 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:32.971712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b77f-f3e6-438d-8c29-8e4cb6668527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:32.972048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b77f-f3e6-438d-8c29-8e4cb6668527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:32.972651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05b77f-f3e6-438d-8c29-8e4cb6668527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:32.974274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05b77f-f3e6-438d-8c29-8e4cb6668527 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:32.987914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 769/1505] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:32.990662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df7b61-c070-4001-a0b4-a123f20d7637 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:33.001846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df7b61-c070-4001-a0b4-a123f20d7637 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:33.011670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59b619-c52c-45c4-bfa4-60f1903d217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:33.013101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59b619-c52c-45c4-bfa4-60f1903d217e admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:33.013101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59b619-c52c-45c4-bfa4-60f1903d217e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:33.013477 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59b619-c52c-45c4-bfa4-60f1903d217e admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:33.014203 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f59b619-c52c-45c4-bfa4-60f1903d217e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:33.017780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f59b619-c52c-45c4-bfa4-60f1903d217e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:33.046257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 737/1506] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:33.052140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b77f-f3e6-438d-8c29-8e4cb6668527 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:33.067856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b77f-f3e6-438d-8c29-8e4cb6668527 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:33.080292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416ae15-2205-4769-a293-a139a019a1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:33.082359 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416ae15-2205-4769-a293-a139a019a1d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:33.082936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416ae15-2205-4769-a293-a139a019a1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:33.083387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416ae15-2205-4769-a293-a139a019a1d0 admin admin] PATH_INFO: `/v3/projects/d9580e20328e4463971a94429e1d07db/users/c1de5ec82c5640caba4636a2286ec989/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:33.116561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416ae15-2205-4769-a293-a139a019a1d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9580e20328e4463971a94429e1d07db, user_id=c1de5ec82c5640caba4636a2286ec989, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:33.124944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416ae15-2205-4769-a293-a139a019a1d0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:33.159119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 770/1507] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:33 2022] PUT /identity/v3/projects/d9580e20328e4463971a94429e1d07db/users/c1de5ec82c5640caba4636a2286ec989/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:33.185324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c75dd-1779-48b3-a090-666368b32ae6 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:33.185324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c75dd-1779-48b3-a090-666368b32ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:33.185324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c75dd-1779-48b3-a090-666368b32ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:33.267497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-269c75dd-1779-48b3-a090-666368b32ae6 None None] MFA Rules not processed for user `c1de5ec82c5640caba4636a2286ec989`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:33.270065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-269c75dd-1779-48b3-a090-666368b32ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:33.354661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:33.368895 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 738/1508] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:33.370176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416ae15-2205-4769-a293-a139a019a1d0 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:33.455805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:33.514952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416ae15-2205-4769-a293-a139a019a1d0 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:33.520524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd158f6-e079-45b6-b5bc-9e85717ac432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:33.521570 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd158f6-e079-45b6-b5bc-9e85717ac432 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:33.521911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd158f6-e079-45b6-b5bc-9e85717ac432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:33.522239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd158f6-e079-45b6-b5bc-9e85717ac432 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:33.525742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd158f6-e079-45b6-b5bc-9e85717ac432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:33.527714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd158f6-e079-45b6-b5bc-9e85717ac432 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:33.588663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 771/1509] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:33 2022] POST /identity/v3/projects => generated 371 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:55:33.592523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c75dd-1779-48b3-a090-666368b32ae6 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:33.612939 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c75dd-1779-48b3-a090-666368b32ae6 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:33.684201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35726ba-dd07-4039-b874-9fd53f6d650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:33.685043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35726ba-dd07-4039-b874-9fd53f6d650c admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:33.685537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35726ba-dd07-4039-b874-9fd53f6d650c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:33.685884 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35726ba-dd07-4039-b874-9fd53f6d650c admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:33.686598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35726ba-dd07-4039-b874-9fd53f6d650c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:33.688667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35726ba-dd07-4039-b874-9fd53f6d650c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:33.716365 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 739/1510] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:33 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:55:33.720334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd158f6-e079-45b6-b5bc-9e85717ac432 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:33.732374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd158f6-e079-45b6-b5bc-9e85717ac432 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:33.736980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d1a7f-7ec6-45f6-a829-171db844a0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:33.737573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d1a7f-7ec6-45f6-a829-171db844a0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:33.737846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d1a7f-7ec6-45f6-a829-171db844a0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:33.738111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d1a7f-7ec6-45f6-a829-171db844a0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:33.738753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d1a7f-7ec6-45f6-a829-171db844a0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:33.740401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d1a7f-7ec6-45f6-a829-171db844a0d6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:33.755915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 772/1511] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:33.759260 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35726ba-dd07-4039-b874-9fd53f6d650c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:33.769907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35726ba-dd07-4039-b874-9fd53f6d650c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:33.777223 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ddd4d-41e4-4e86-9d0a-c72dd6c55672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:33.777940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ddd4d-41e4-4e86-9d0a-c72dd6c55672 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:33.778218 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ddd4d-41e4-4e86-9d0a-c72dd6c55672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:33.778491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ddd4d-41e4-4e86-9d0a-c72dd6c55672 admin admin] PATH_INFO: `/v3/projects/a4fca65f60714371a6bc236d60ef1ecc/users/7bb9d7ef507e48709085616f4661c11a/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:33.795242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ddd4d-41e4-4e86-9d0a-c72dd6c55672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4fca65f60714371a6bc236d60ef1ecc, user_id=7bb9d7ef507e48709085616f4661c11a, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:33.797993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ddd4d-41e4-4e86-9d0a-c72dd6c55672 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:33.815075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 740/1512] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:33 2022] PUT /identity/v3/projects/a4fca65f60714371a6bc236d60ef1ecc/users/7bb9d7ef507e48709085616f4661c11a/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:33.821120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d1a7f-7ec6-45f6-a829-171db844a0d6 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:33.833073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d1a7f-7ec6-45f6-a829-171db844a0d6 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:33.902543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454c746-1f5f-452e-8ab4-c95e01832741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:33.903138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c746-1f5f-452e-8ab4-c95e01832741 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:33.903501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c746-1f5f-452e-8ab4-c95e01832741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:33.903750 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c746-1f5f-452e-8ab4-c95e01832741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:33.904463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454c746-1f5f-452e-8ab4-c95e01832741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:33.906211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454c746-1f5f-452e-8ab4-c95e01832741 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:33.924327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 773/1513] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:33.928694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ddd4d-41e4-4e86-9d0a-c72dd6c55672 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:33.948801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ddd4d-41e4-4e86-9d0a-c72dd6c55672 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:33.956229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b3b28-ccf4-4ce4-8af4-09d6cb715e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:33.956962 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b3b28-ccf4-4ce4-8af4-09d6cb715e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:33.957265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b3b28-ccf4-4ce4-8af4-09d6cb715e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:33.957564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b3b28-ccf4-4ce4-8af4-09d6cb715e95 admin admin] PATH_INFO: `/v3/projects/a4fca65f60714371a6bc236d60ef1ecc/users/7bb9d7ef507e48709085616f4661c11a/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:33.961227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b3b28-ccf4-4ce4-8af4-09d6cb715e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4fca65f60714371a6bc236d60ef1ecc, user_id=7bb9d7ef507e48709085616f4661c11a, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:33.962857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b3b28-ccf4-4ce4-8af4-09d6cb715e95 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:33.998427 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 741/1514] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:33 2022] PUT /identity/v3/projects/a4fca65f60714371a6bc236d60ef1ecc/users/7bb9d7ef507e48709085616f4661c11a/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:34.012605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7a586-f41c-401d-a551-6ddb0dcc6c02 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:34.012886 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7a586-f41c-401d-a551-6ddb0dcc6c02 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:34.013161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7a586-f41c-401d-a551-6ddb0dcc6c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:34.063113 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-85d7a586-f41c-401d-a551-6ddb0dcc6c02 None None] MFA Rules not processed for user `7bb9d7ef507e48709085616f4661c11a`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:34.064503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-85d7a586-f41c-401d-a551-6ddb0dcc6c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:34.144829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 774/1515] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:34.157312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b3b28-ccf4-4ce4-8af4-09d6cb715e95 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:34.159410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-560b3b28-ccf4-4ce4-8af4-09d6cb715e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:34.250877 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b3b28-ccf4-4ce4-8af4-09d6cb715e95 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:34.256232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea04c4-50df-4e1f-97d7-cba9368e9781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:34.257153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea04c4-50df-4e1f-97d7-cba9368e9781 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:34.257582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea04c4-50df-4e1f-97d7-cba9368e9781 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:34.258722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea04c4-50df-4e1f-97d7-cba9368e9781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:34.271695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea04c4-50df-4e1f-97d7-cba9368e9781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:34.275243 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea04c4-50df-4e1f-97d7-cba9368e9781 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:34.291412 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 742/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:34.963026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7a586-f41c-401d-a551-6ddb0dcc6c02 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:35.006078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:35.045396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7a586-f41c-401d-a551-6ddb0dcc6c02 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:35.056737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:35.057464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:35.057749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:35.058032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] PATH_INFO: `/v3/users/c1de5ec82c5640caba4636a2286ec989` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:35.059173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1de5ec82c5640caba4636a2286ec989)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:35.061184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:35.114756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1de5ec82c5640caba4636a2286ec989'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:35.115984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1de5ec82c5640caba4636a2286ec989'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:35.124531 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1de5ec82c5640caba4636a2286ec989'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:35.125262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1de5ec82c5640caba4636a2286ec989'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:35.142392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1de5ec82c5640caba4636a2286ec989'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:35.159256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 775/1517] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:34 2022] DELETE /identity/v3/users/c1de5ec82c5640caba4636a2286ec989 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:35.165193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50b33f-3b0a-4514-8b7c-c522c387497d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:35.165458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50b33f-3b0a-4514-8b7c-c522c387497d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:35.165926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50b33f-3b0a-4514-8b7c-c522c387497d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:35.186647 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-3d50b33f-3b0a-4514-8b7c-c522c387497d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:35.188099 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3d50b33f-3b0a-4514-8b7c-c522c387497d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:35.241686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:35.262968 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 743/1518] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:35.272402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:35.274058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:35.362639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa272eb-900b-48af-b62e-7a9199d09846 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:35.366225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92232db5-8b06-4a3e-a753-bc3171690a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:35.366835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232db5-8b06-4a3e-a753-bc3171690a7e service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:35.367635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232db5-8b06-4a3e-a753-bc3171690a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:35.368194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232db5-8b06-4a3e-a753-bc3171690a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:35.378354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92232db5-8b06-4a3e-a753-bc3171690a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:35.379815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92232db5-8b06-4a3e-a753-bc3171690a7e service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:35.391051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 776/1519] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:35.702335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50b33f-3b0a-4514-8b7c-c522c387497d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:35.705491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3d50b33f-3b0a-4514-8b7c-c522c387497d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:35.749812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50b33f-3b0a-4514-8b7c-c522c387497d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:35.765793 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:35.766854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:35.767433 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:35.767953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] PATH_INFO: `/v3/projects/d9580e20328e4463971a94429e1d07db` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:35.772513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9580e20328e4463971a94429e1d07db)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:35.775735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:35.889446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9580e20328e4463971a94429e1d07db', 'request_id': 'req-b5f71f80-3081-4129-827c-d919e3c10bee'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:35.899999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9580e20328e4463971a94429e1d07db', 'request_id': 'req-b5f71f80-3081-4129-827c-d919e3c10bee'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:35.912407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9580e20328e4463971a94429e1d07db', 'request_id': 'req-b5f71f80-3081-4129-827c-d919e3c10bee'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:35.929313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] The token cache is being invalidate because project d9580e20328e4463971a94429e1d07db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:35.929600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:35.931219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:35.935778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 744/1520] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:35 2022] DELETE /identity/v3/projects/d9580e20328e4463971a94429e1d07db => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:35.936861 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92232db5-8b06-4a3e-a753-bc3171690a7e service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:35.947653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92232db5-8b06-4a3e-a753-bc3171690a7e service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:36.009715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:36.024914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:36.025544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:36.025817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:36.026083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] PATH_INFO: `/v3/users/7bb9d7ef507e48709085616f4661c11a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:36.027226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bb9d7ef507e48709085616f4661c11a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:36.029166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:36.078791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bb9d7ef507e48709085616f4661c11a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:36.079418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bb9d7ef507e48709085616f4661c11a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:36.087965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bb9d7ef507e48709085616f4661c11a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:36.088685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bb9d7ef507e48709085616f4661c11a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:36.101199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7e40ee84-0249-477c-b6ed-afd6acfd5057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bb9d7ef507e48709085616f4661c11a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:36.118195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 777/1521] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:35 2022] DELETE /identity/v3/users/7bb9d7ef507e48709085616f4661c11a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:36.489847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:36.491523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:36.515053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f71f80-3081-4129-827c-d919e3c10bee admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:36.588579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:36.589182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:36.589462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:36.589733 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] PATH_INFO: `/v3/projects/a4fca65f60714371a6bc236d60ef1ecc` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:36.590915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4fca65f60714371a6bc236d60ef1ecc)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:36.592891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:36.657399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4fca65f60714371a6bc236d60ef1ecc', 'request_id': 'req-8062313b-72dc-410f-bcc5-f51048e82f67'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:36.669690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4fca65f60714371a6bc236d60ef1ecc', 'request_id': 'req-8062313b-72dc-410f-bcc5-f51048e82f67'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:36.681750 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4fca65f60714371a6bc236d60ef1ecc', 'request_id': 'req-8062313b-72dc-410f-bcc5-f51048e82f67'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:36.703303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] The token cache is being invalidate because project a4fca65f60714371a6bc236d60ef1ecc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:36.703912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:36.705254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8062313b-72dc-410f-bcc5-f51048e82f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:36.708159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 745/1522] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:36 2022] DELETE /identity/v3/projects/a4fca65f60714371a6bc236d60ef1ecc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:36.714795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fed93-c563-4929-a7d7-42fe0983cf96 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:36.715154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fed93-c563-4929-a7d7-42fe0983cf96 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:36.715506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fed93-c563-4929-a7d7-42fe0983cf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:36.740399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ee2fed93-c563-4929-a7d7-42fe0983cf96 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:36.741989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ee2fed93-c563-4929-a7d7-42fe0983cf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:36.826486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 778/1523] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:36.837240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bb870-9da1-4b1d-83bc-7bbd8ae1d2c3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:36.837843 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bb870-9da1-4b1d-83bc-7bbd8ae1d2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:36.838318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bb870-9da1-4b1d-83bc-7bbd8ae1d2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:36.858532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:36.875836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-795bb870-9da1-4b1d-83bc-7bbd8ae1d2c3 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:36.875836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-795bb870-9da1-4b1d-83bc-7bbd8ae1d2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:36.900608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 746/1524] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:36.904832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fed93-c563-4929-a7d7-42fe0983cf96 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:36.918483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fed93-c563-4929-a7d7-42fe0983cf96 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:36.924904 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1ca76-1f28-4d2b-9ccf-7b01cfee227f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:36.925663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1ca76-1f28-4d2b-9ccf-7b01cfee227f admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:36.926032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1ca76-1f28-4d2b-9ccf-7b01cfee227f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:36.926583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1ca76-1f28-4d2b-9ccf-7b01cfee227f admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:36.927496 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1ca76-1f28-4d2b-9ccf-7b01cfee227f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:36.927924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1ca76-1f28-4d2b-9ccf-7b01cfee227f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:36.929648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1ca76-1f28-4d2b-9ccf-7b01cfee227f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:36.954692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 779/1525] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:36.959215 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bb870-9da1-4b1d-83bc-7bbd8ae1d2c3 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:36.969243 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bb870-9da1-4b1d-83bc-7bbd8ae1d2c3 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:36.977471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa76784-820f-498e-b60c-8b93ccab93db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:36.978179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa76784-820f-498e-b60c-8b93ccab93db admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:36.978348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa76784-820f-498e-b60c-8b93ccab93db admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:36.978621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa76784-820f-498e-b60c-8b93ccab93db admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:36.979263 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa76784-820f-498e-b60c-8b93ccab93db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:36.981273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa76784-820f-498e-b60c-8b93ccab93db admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:37.049549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 747/1526] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:36 2022] POST /identity/v3/projects => generated 367 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:37.053716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1ca76-1f28-4d2b-9ccf-7b01cfee227f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:37.073341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1ca76-1f28-4d2b-9ccf-7b01cfee227f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:37.143367 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:37.153431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fea807-dd35-4e7e-8e4e-56055850c335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:37.154311 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea807-dd35-4e7e-8e4e-56055850c335 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:37.155088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea807-dd35-4e7e-8e4e-56055850c335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:37.155359 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea807-dd35-4e7e-8e4e-56055850c335 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:37.156714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fea807-dd35-4e7e-8e4e-56055850c335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:37.158872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fea807-dd35-4e7e-8e4e-56055850c335 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:37.193664 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 780/1527] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:37 2022] POST /identity/v3/users => generated 284 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:37.197727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa76784-820f-498e-b60c-8b93ccab93db admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:37.211971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa76784-820f-498e-b60c-8b93ccab93db admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:37.216723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24d207-54d7-465a-aaa1-851ffec16b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:37.217329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24d207-54d7-465a-aaa1-851ffec16b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:37.217604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24d207-54d7-465a-aaa1-851ffec16b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:37.217868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24d207-54d7-465a-aaa1-851ffec16b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:37.218510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24d207-54d7-465a-aaa1-851ffec16b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:37.220133 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24d207-54d7-465a-aaa1-851ffec16b8c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:37.232825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 748/1528] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:55:37.236746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fea807-dd35-4e7e-8e4e-56055850c335 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:37.248041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fea807-dd35-4e7e-8e4e-56055850c335 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:37.253035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f3ad5-e6be-434d-b386-b2647b464f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:37.253763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f3ad5-e6be-434d-b386-b2647b464f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:37.254136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f3ad5-e6be-434d-b386-b2647b464f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:37.254513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f3ad5-e6be-434d-b386-b2647b464f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:37.256342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f3ad5-e6be-434d-b386-b2647b464f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:37.258344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f3ad5-e6be-434d-b386-b2647b464f5c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:37.271920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 781/1529] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:37.275264 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24d207-54d7-465a-aaa1-851ffec16b8c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:37.286314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24d207-54d7-465a-aaa1-851ffec16b8c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:37.292988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1967d44-bbaf-4962-9a8d-f915fac39d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:37.293780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1967d44-bbaf-4962-9a8d-f915fac39d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:37.294154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1967d44-bbaf-4962-9a8d-f915fac39d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:37.294689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1967d44-bbaf-4962-9a8d-f915fac39d6f admin admin] PATH_INFO: `/v3/projects/368f4c071772497aa3811eb2dd080a35/users/cd07c81819f44e059b72a9fa32f03b02/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:37.312264 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1967d44-bbaf-4962-9a8d-f915fac39d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=368f4c071772497aa3811eb2dd080a35, user_id=cd07c81819f44e059b72a9fa32f03b02, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:37.315362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1967d44-bbaf-4962-9a8d-f915fac39d6f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:37.334364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 749/1530] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:37 2022] PUT /identity/v3/projects/368f4c071772497aa3811eb2dd080a35/users/cd07c81819f44e059b72a9fa32f03b02/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:37.345848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bb3c1-432e-4ead-8984-2b6af8546a97 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:37.346130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bb3c1-432e-4ead-8984-2b6af8546a97 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:37.346432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bb3c1-432e-4ead-8984-2b6af8546a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:37.396808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9b8bb3c1-432e-4ead-8984-2b6af8546a97 None None] MFA Rules not processed for user `cd07c81819f44e059b72a9fa32f03b02`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:37.398047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9b8bb3c1-432e-4ead-8984-2b6af8546a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:37.491147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 782/1531] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:37.495523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1967d44-bbaf-4962-9a8d-f915fac39d6f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:37.581920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bb3c1-432e-4ead-8984-2b6af8546a97 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:37.584877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9b8bb3c1-432e-4ead-8984-2b6af8546a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:37.625000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1967d44-bbaf-4962-9a8d-f915fac39d6f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:37.630604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df4f3d-a70f-46e1-9c6b-e18b96b8bcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:37.631413 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4f3d-a70f-46e1-9c6b-e18b96b8bcbb admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:37.631745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4f3d-a70f-46e1-9c6b-e18b96b8bcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:37.632246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4f3d-a70f-46e1-9c6b-e18b96b8bcbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:37.633966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df4f3d-a70f-46e1-9c6b-e18b96b8bcbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:37.639236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df4f3d-a70f-46e1-9c6b-e18b96b8bcbb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:37.688389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 750/1532] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:37 2022] POST /identity/v3/projects => generated 367 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:37.691682 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df4f3d-a70f-46e1-9c6b-e18b96b8bcbb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:37.701629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bb3c1-432e-4ead-8984-2b6af8546a97 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:37.771775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bea5a-13d1-44af-b8c5-207f172b5d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:37.772438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bea5a-13d1-44af-b8c5-207f172b5d96 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:37.772718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bea5a-13d1-44af-b8c5-207f172b5d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:37.772984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bea5a-13d1-44af-b8c5-207f172b5d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:37.776125 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c64bea5a-13d1-44af-b8c5-207f172b5d96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:37.830991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:37.833663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df4f3d-a70f-46e1-9c6b-e18b96b8bcbb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:37.849370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62ecb9-aa13-41b8-bf2a-3e4dad55c4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:37.850066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62ecb9-aa13-41b8-bf2a-3e4dad55c4be admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:37.850406 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62ecb9-aa13-41b8-bf2a-3e4dad55c4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:37.850824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62ecb9-aa13-41b8-bf2a-3e4dad55c4be admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:37.852511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62ecb9-aa13-41b8-bf2a-3e4dad55c4be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:37.859923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62ecb9-aa13-41b8-bf2a-3e4dad55c4be admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:37.889712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bea5a-13d1-44af-b8c5-207f172b5d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:37.891415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bea5a-13d1-44af-b8c5-207f172b5d96 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:37.902549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 751/1533] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:37 2022] POST /identity/v3/users => generated 290 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:55:37.905084 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62ecb9-aa13-41b8-bf2a-3e4dad55c4be admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:37.909526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 783/1534] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:55:37.924667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62ecb9-aa13-41b8-bf2a-3e4dad55c4be admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:37.932198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63d722-0c3d-434d-b769-41ee29d9f7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:37.933565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63d722-0c3d-434d-b769-41ee29d9f7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:37.933873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63d722-0c3d-434d-b769-41ee29d9f7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:37.934122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63d722-0c3d-434d-b769-41ee29d9f7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:37.934819 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63d722-0c3d-434d-b769-41ee29d9f7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:37.936831 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63d722-0c3d-434d-b769-41ee29d9f7d3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:37.963917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 752/1535] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:37.967955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bea5a-13d1-44af-b8c5-207f172b5d96 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:37.988202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bea5a-13d1-44af-b8c5-207f172b5d96 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:37.995832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebc920-34d1-4e78-a1ba-50122dfe20ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:37.996630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebc920-34d1-4e78-a1ba-50122dfe20ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:37.997332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebc920-34d1-4e78-a1ba-50122dfe20ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:38.004298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebc920-34d1-4e78-a1ba-50122dfe20ce admin admin] PATH_INFO: `/v3/projects/f13a487274cf4accac4b65eb022c6201/users/2160c4cd63294e6683e85932233d8c1f/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:38.245909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebc920-34d1-4e78-a1ba-50122dfe20ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13a487274cf4accac4b65eb022c6201, user_id=2160c4cd63294e6683e85932233d8c1f, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:38.250187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebc920-34d1-4e78-a1ba-50122dfe20ce admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:38.282288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63d722-0c3d-434d-b769-41ee29d9f7d3 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:38.282559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 784/1536] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:37 2022] PUT /identity/v3/projects/f13a487274cf4accac4b65eb022c6201/users/2160c4cd63294e6683e85932233d8c1f/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:38.319559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63d722-0c3d-434d-b769-41ee29d9f7d3 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:38.335737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:38.476018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e676c-2e3e-48b7-a280-7baf1c114132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:38.476969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e676c-2e3e-48b7-a280-7baf1c114132 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:38.477244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e676c-2e3e-48b7-a280-7baf1c114132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:38.477513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e676c-2e3e-48b7-a280-7baf1c114132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:38.478675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e676c-2e3e-48b7-a280-7baf1c114132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:38.480375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e676c-2e3e-48b7-a280-7baf1c114132 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:38.492242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 753/1537] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:38.496991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebc920-34d1-4e78-a1ba-50122dfe20ce admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:38.525064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebc920-34d1-4e78-a1ba-50122dfe20ce admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:38.534435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66299c8-99db-416c-b512-0435c8b458f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:38.535119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66299c8-99db-416c-b512-0435c8b458f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:38.535394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66299c8-99db-416c-b512-0435c8b458f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:38.535718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66299c8-99db-416c-b512-0435c8b458f1 admin admin] PATH_INFO: `/v3/projects/f13a487274cf4accac4b65eb022c6201/users/2160c4cd63294e6683e85932233d8c1f/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:38.537648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66299c8-99db-416c-b512-0435c8b458f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13a487274cf4accac4b65eb022c6201, user_id=2160c4cd63294e6683e85932233d8c1f, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:38.540436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66299c8-99db-416c-b512-0435c8b458f1 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:38.601432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 785/1538] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:38 2022] PUT /identity/v3/projects/f13a487274cf4accac4b65eb022c6201/users/2160c4cd63294e6683e85932233d8c1f/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:38.605053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396708b7-b236-427c-a7f6-26f3a732cf6e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:38.605737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396708b7-b236-427c-a7f6-26f3a732cf6e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:38.606029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396708b7-b236-427c-a7f6-26f3a732cf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:38.676735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-396708b7-b236-427c-a7f6-26f3a732cf6e None None] MFA Rules not processed for user `2160c4cd63294e6683e85932233d8c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:38.678366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-396708b7-b236-427c-a7f6-26f3a732cf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:38.787966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 754/1539] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:38.796389 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66299c8-99db-416c-b512-0435c8b458f1 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:38.881316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66299c8-99db-416c-b512-0435c8b458f1 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:38.897885 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f9a84-052b-4cf2-ab1d-16207ff57bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:38.898642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9a84-052b-4cf2-ab1d-16207ff57bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:38.899356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9a84-052b-4cf2-ab1d-16207ff57bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:38.899356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9a84-052b-4cf2-ab1d-16207ff57bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:38.912542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f9a84-052b-4cf2-ab1d-16207ff57bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:38.914358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f9a84-052b-4cf2-ab1d-16207ff57bf9 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:38.939496 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 786/1540] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:39.406348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396708b7-b236-427c-a7f6-26f3a732cf6e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:39.408079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-396708b7-b236-427c-a7f6-26f3a732cf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:39.511481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396708b7-b236-427c-a7f6-26f3a732cf6e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:39.524152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:39.524916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:39.525338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:39.525615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] PATH_INFO: `/v3/users/db60633510b041fb9008105119fee5ba` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:39.526958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db60633510b041fb9008105119fee5ba)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:39.530914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:39.621921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:39.646288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db60633510b041fb9008105119fee5ba'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:39.646980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db60633510b041fb9008105119fee5ba'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:39.655036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db60633510b041fb9008105119fee5ba'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:39.655928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db60633510b041fb9008105119fee5ba'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:39.696852 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db60633510b041fb9008105119fee5ba'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:39.737399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 755/1541] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:39 2022] DELETE /identity/v3/users/db60633510b041fb9008105119fee5ba => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:39.743002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bb680-00b6-4e07-ba03-e1f53e6c7b36 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:39.743731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bb680-00b6-4e07-ba03-e1f53e6c7b36 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:39.743731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bb680-00b6-4e07-ba03-e1f53e6c7b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:39.793521 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ee1bb680-00b6-4e07-ba03-e1f53e6c7b36 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:39.795983 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ee1bb680-00b6-4e07-ba03-e1f53e6c7b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:39.990079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 787/1542] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:39.999535 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:40.002866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:40.218722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f17551-6089-4f59-8a9a-7117c016feb1 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:40.226153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1f4ad-ed05-45e0-a62a-d1d6704c2ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:40.227838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1f4ad-ed05-45e0-a62a-d1d6704c2ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:40.228417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1f4ad-ed05-45e0-a62a-d1d6704c2ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:40.228965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1f4ad-ed05-45e0-a62a-d1d6704c2ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:40.249723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1f4ad-ed05-45e0-a62a-d1d6704c2ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:40.252852 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1f4ad-ed05-45e0-a62a-d1d6704c2ab7 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:40.282383 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 756/1543] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:40.620503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bb680-00b6-4e07-ba03-e1f53e6c7b36 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:40.622075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ee1bb680-00b6-4e07-ba03-e1f53e6c7b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:40.635953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bb680-00b6-4e07-ba03-e1f53e6c7b36 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:40.641987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:40.643312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:40.643999 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:40.644693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] PATH_INFO: `/v3/projects/49aafe383fa141e796cff32be0842b82` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:40.646202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=49aafe383fa141e796cff32be0842b82)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:40.648731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:40.687294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:40.704312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49aafe383fa141e796cff32be0842b82', 'request_id': 'req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:40.712254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49aafe383fa141e796cff32be0842b82', 'request_id': 'req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:40.721072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49aafe383fa141e796cff32be0842b82', 'request_id': 'req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:40.740481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] The token cache is being invalidate because project 49aafe383fa141e796cff32be0842b82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:40.741780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:40.743266 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbe1d5b-0d33-4435-8af8-c22b9b0fbb0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:40.745306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 788/1544] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:40 2022] DELETE /identity/v3/projects/49aafe383fa141e796cff32be0842b82 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:40.748999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1f4ad-ed05-45e0-a62a-d1d6704c2ab7 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:40.766745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1f4ad-ed05-45e0-a62a-d1d6704c2ab7 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:40.862071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:40.862799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:40.863147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:40.863949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] PATH_INFO: `/v3/users/b3494c84f3f842aa8149cd3f104d6e02` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:40.865555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3494c84f3f842aa8149cd3f104d6e02)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:40.869148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:40.924672 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3494c84f3f842aa8149cd3f104d6e02'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:40.925431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3494c84f3f842aa8149cd3f104d6e02'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:40.933203 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:40.935900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3494c84f3f842aa8149cd3f104d6e02'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:40.936867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3494c84f3f842aa8149cd3f104d6e02'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:40.955784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3494c84f3f842aa8149cd3f104d6e02'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:40.978764 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 757/1545] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:40 2022] DELETE /identity/v3/users/b3494c84f3f842aa8149cd3f104d6e02 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:41.158345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69592945-4927-4540-957b-e76810c7d502 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:41.158630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69592945-4927-4540-957b-e76810c7d502 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:41.158899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69592945-4927-4540-957b-e76810c7d502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:41.229784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-69592945-4927-4540-957b-e76810c7d502 None None] MFA Rules not processed for user `3c17f274e4c14f2f99650844642437a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:41.231089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-69592945-4927-4540-957b-e76810c7d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:41.372639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:41.374761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:41.441296 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 789/1546] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Oct 14 17:55:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:41.442277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69592945-4927-4540-957b-e76810c7d502 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:41.443818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-69592945-4927-4540-957b-e76810c7d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:41.530895 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473f95c-de26-4c9b-8115-53bf4e3ea59f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:41.538078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:41.539143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:41.540115 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:41.540534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] PATH_INFO: `/v3/users/cd07c81819f44e059b72a9fa32f03b02` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:41.542275 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd07c81819f44e059b72a9fa32f03b02)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:41.545638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:41.576143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69592945-4927-4540-957b-e76810c7d502 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:41.595342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:41.596033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:41.596310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:41.596579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] PATH_INFO: `/v3/projects/1500cdd38e944ea280fd55a9c30ab914` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:41.603450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1500cdd38e944ea280fd55a9c30ab914)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:41.605685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:41.648486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd07c81819f44e059b72a9fa32f03b02'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:41.650492 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd07c81819f44e059b72a9fa32f03b02'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:41.660438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd07c81819f44e059b72a9fa32f03b02'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:41.661472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd07c81819f44e059b72a9fa32f03b02'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:41.677189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd07c81819f44e059b72a9fa32f03b02'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:41.692114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1500cdd38e944ea280fd55a9c30ab914', 'request_id': 'req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:41.705124 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 758/1547] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:41 2022] DELETE /identity/v3/users/cd07c81819f44e059b72a9fa32f03b02 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:41.706092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:41.707548 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:41.712537 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1500cdd38e944ea280fd55a9c30ab914', 'request_id': 'req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:41.729403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1500cdd38e944ea280fd55a9c30ab914', 'request_id': 'req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:41.744855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:41.750834 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] The token cache is being invalidate because project 1500cdd38e944ea280fd55a9c30ab914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:41.751132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:41.752433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:41.755082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 790/1548] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:41 2022] DELETE /identity/v3/projects/1500cdd38e944ea280fd55a9c30ab914 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:41.756659 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:41.759111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:41.791055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:41.795734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb023e7-d3cb-4bf9-870b-2fa8a1ff0d16 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:41.802289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf48e9f-d94e-4430-903e-5ab6f0cbd9f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:41.802989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48e9f-d94e-4430-903e-5ab6f0cbd9f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:41.803610 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48e9f-d94e-4430-903e-5ab6f0cbd9f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:41.804033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48e9f-d94e-4430-903e-5ab6f0cbd9f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:41.812550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-edf48e9f-d94e-4430-903e-5ab6f0cbd9f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:41.827490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf48e9f-d94e-4430-903e-5ab6f0cbd9f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:41.829298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf48e9f-d94e-4430-903e-5ab6f0cbd9f3 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:41.835812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34ba88-d4f9-4992-9a66-9ca2fdededd2 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:41.838996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 759/1549] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:41.840409 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b305dc-6d1e-41ba-b644-1d7f519b30e2 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:41.840959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b305dc-6d1e-41ba-b644-1d7f519b30e2 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:41.841375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b305dc-6d1e-41ba-b644-1d7f519b30e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:41.846832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e58fbc-d02b-4d82-abcd-040d2a51e5c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:41.853117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e58fbc-d02b-4d82-abcd-040d2a51e5c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:41.853554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e58fbc-d02b-4d82-abcd-040d2a51e5c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:41.853829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e58fbc-d02b-4d82-abcd-040d2a51e5c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:41.864647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e58fbc-d02b-4d82-abcd-040d2a51e5c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:41.866197 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e58fbc-d02b-4d82-abcd-040d2a51e5c7 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:41.868663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f9b305dc-6d1e-41ba-b644-1d7f519b30e2 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:41.870052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f9b305dc-6d1e-41ba-b644-1d7f519b30e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:41.895563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 791/1550] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:41.897235 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d13f22-05fa-4a4e-9f50-33189a011709 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:41.897827 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d13f22-05fa-4a4e-9f50-33189a011709 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:41.898350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d13f22-05fa-4a4e-9f50-33189a011709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:41.949170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-35d13f22-05fa-4a4e-9f50-33189a011709 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:41.950431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-35d13f22-05fa-4a4e-9f50-33189a011709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:41.983066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 760/1551] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:41.986222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b305dc-6d1e-41ba-b644-1d7f519b30e2 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:41.987396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f9b305dc-6d1e-41ba-b644-1d7f519b30e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:42.007785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b305dc-6d1e-41ba-b644-1d7f519b30e2 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:42.022736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:42.023428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.024035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.024035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] PATH_INFO: `/v3/users/d680d547ec9043d48ab6cec6dcd08468` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.026633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d680d547ec9043d48ab6cec6dcd08468)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:42.030443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:42.048307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 792/1552] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:42.049647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d13f22-05fa-4a4e-9f50-33189a011709 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:42.068024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d13f22-05fa-4a4e-9f50-33189a011709 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:42.074509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233aaf0c-c271-414a-87d8-700d204d11ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:42.075381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233aaf0c-c271-414a-87d8-700d204d11ed service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.075775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233aaf0c-c271-414a-87d8-700d204d11ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.076593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233aaf0c-c271-414a-87d8-700d204d11ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.092701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233aaf0c-c271-414a-87d8-700d204d11ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:42.094618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233aaf0c-c271-414a-87d8-700d204d11ed service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:42.112157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 793/1553] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:42.114168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebb690-ca6d-4ff9-b7db-a55c0efda05c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.114462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd680d547ec9043d48ab6cec6dcd08468'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:42.114637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebb690-ca6d-4ff9-b7db-a55c0efda05c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.115108 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebb690-ca6d-4ff9-b7db-a55c0efda05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.115314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd680d547ec9043d48ab6cec6dcd08468'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:42.128834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd680d547ec9043d48ab6cec6dcd08468'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:42.129990 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd680d547ec9043d48ab6cec6dcd08468'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:42.147414 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e71963b6-7ae1-484b-a778-c8bcd45c014b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd680d547ec9043d48ab6cec6dcd08468'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:42.176054 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e5ebb690-ca6d-4ff9-b7db-a55c0efda05c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:42.176054 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e5ebb690-ca6d-4ff9-b7db-a55c0efda05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:42.176370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 761/1554] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:41 2022] DELETE /identity/v3/users/d680d547ec9043d48ab6cec6dcd08468 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:42.176370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac31531-aceb-4680-89a6-3b094708fbeb None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.176370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac31531-aceb-4680-89a6-3b094708fbeb None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.176370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac31531-aceb-4680-89a6-3b094708fbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.177971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 794/1555] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:42.182819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebb690-ca6d-4ff9-b7db-a55c0efda05c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:42.204984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebb690-ca6d-4ff9-b7db-a55c0efda05c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:42.207418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-9ac31531-aceb-4680-89a6-3b094708fbeb None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:42.208812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9ac31531-aceb-4680-89a6-3b094708fbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:42.338624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36763534-089c-47d6-be69-08dacc25bfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:42.339259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763534-089c-47d6-be69-08dacc25bfe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.339686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763534-089c-47d6-be69-08dacc25bfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.340436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763534-089c-47d6-be69-08dacc25bfe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.341197 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36763534-089c-47d6-be69-08dacc25bfe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:42.341550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36763534-089c-47d6-be69-08dacc25bfe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:42.344333 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36763534-089c-47d6-be69-08dacc25bfe4 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:42.363259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 795/1556] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:55:42.369855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36763534-089c-47d6-be69-08dacc25bfe4 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:42.376763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 762/1557] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:42.385106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36763534-089c-47d6-be69-08dacc25bfe4 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:42.392165 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79365d53-f0a2-41f3-958d-c5957581fa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:42.392373 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79365d53-f0a2-41f3-958d-c5957581fa9d admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.392677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79365d53-f0a2-41f3-958d-c5957581fa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.392928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79365d53-f0a2-41f3-958d-c5957581fa9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.393717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79365d53-f0a2-41f3-958d-c5957581fa9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:42.395994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79365d53-f0a2-41f3-958d-c5957581fa9d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:42.421758 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:42.451132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac31531-aceb-4680-89a6-3b094708fbeb None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:42.452607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9ac31531-aceb-4680-89a6-3b094708fbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:42.459313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:42.465141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 796/1558] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:42 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:42.466711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79365d53-f0a2-41f3-958d-c5957581fa9d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:42.589172 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac31531-aceb-4680-89a6-3b094708fbeb None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:42.591899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79365d53-f0a2-41f3-958d-c5957581fa9d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:42.598977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ebab7-5cc7-4d15-854f-3a98b89ab8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:42.599602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ebab7-5cc7-4d15-854f-3a98b89ab8ee admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.599855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ebab7-5cc7-4d15-854f-3a98b89ab8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.600177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ebab7-5cc7-4d15-854f-3a98b89ab8ee admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.600870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ebab7-5cc7-4d15-854f-3a98b89ab8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:42.603309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ebab7-5cc7-4d15-854f-3a98b89ab8ee admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:42.637686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 797/1559] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:42 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:42.642274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ebab7-5cc7-4d15-854f-3a98b89ab8ee admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:42.654384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ebab7-5cc7-4d15-854f-3a98b89ab8ee admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:42.659974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3af773-56fc-4514-aa65-8682770b1c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:42.660893 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3af773-56fc-4514-aa65-8682770b1c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.660893 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3af773-56fc-4514-aa65-8682770b1c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.661214 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3af773-56fc-4514-aa65-8682770b1c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.661876 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3af773-56fc-4514-aa65-8682770b1c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:42.663785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3af773-56fc-4514-aa65-8682770b1c08 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:42.681976 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 798/1560] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:42.682617 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab781ffd-d39d-49d4-acc0-3fbce5e214e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:42.683211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3af773-56fc-4514-aa65-8682770b1c08 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:42.683531 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781ffd-d39d-49d4-acc0-3fbce5e214e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.683941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781ffd-d39d-49d4-acc0-3fbce5e214e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.684958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781ffd-d39d-49d4-acc0-3fbce5e214e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.694827 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3af773-56fc-4514-aa65-8682770b1c08 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:42.699130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab781ffd-d39d-49d4-acc0-3fbce5e214e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:42.700959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10b1f5-bd80-4a84-8d3b-bfef76378534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:42.701406 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab781ffd-d39d-49d4-acc0-3fbce5e214e9 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:42.701988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b1f5-bd80-4a84-8d3b-bfef76378534 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.702438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b1f5-bd80-4a84-8d3b-bfef76378534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.702831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b1f5-bd80-4a84-8d3b-bfef76378534 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.703919 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10b1f5-bd80-4a84-8d3b-bfef76378534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:42.706058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10b1f5-bd80-4a84-8d3b-bfef76378534 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:42.719711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 763/1561] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:55:42.720510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 799/1562] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:42.726974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab781ffd-d39d-49d4-acc0-3fbce5e214e9 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:42.740659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab781ffd-d39d-49d4-acc0-3fbce5e214e9 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:42.748274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ace8dc-7359-4bda-9330-7999ae46a718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:42.748960 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ace8dc-7359-4bda-9330-7999ae46a718 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.749411 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ace8dc-7359-4bda-9330-7999ae46a718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.749516 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ace8dc-7359-4bda-9330-7999ae46a718 admin admin] PATH_INFO: `/v3/projects/144ac59f1f4b4f17ac96f5406cfe9d5e/users/23f77ae09e6e44e1bc68842090d5a328/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.770875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ace8dc-7359-4bda-9330-7999ae46a718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=144ac59f1f4b4f17ac96f5406cfe9d5e, user_id=23f77ae09e6e44e1bc68842090d5a328, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:42.774186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ace8dc-7359-4bda-9330-7999ae46a718 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:42.807612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 764/1563] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:42 2022] PUT /identity/v3/projects/144ac59f1f4b4f17ac96f5406cfe9d5e/users/23f77ae09e6e44e1bc68842090d5a328/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:42.811522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd88c0-2764-4270-a3a3-3e5a91e3bcbe None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:42.812091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd88c0-2764-4270-a3a3-3e5a91e3bcbe None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:42.812169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd88c0-2764-4270-a3a3-3e5a91e3bcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:42.847860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-edfd88c0-2764-4270-a3a3-3e5a91e3bcbe None None] MFA Rules not processed for user `23f77ae09e6e44e1bc68842090d5a328`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:42.849140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-edfd88c0-2764-4270-a3a3-3e5a91e3bcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:42.957302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 800/1564] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:42.962370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ace8dc-7359-4bda-9330-7999ae46a718 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:43.053954 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ace8dc-7359-4bda-9330-7999ae46a718 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:43.061138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5757a7d-857e-4ed9-bf8d-bb214b1b36be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:43.062714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5757a7d-857e-4ed9-bf8d-bb214b1b36be admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:43.062714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5757a7d-857e-4ed9-bf8d-bb214b1b36be admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:43.062714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5757a7d-857e-4ed9-bf8d-bb214b1b36be admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:43.064105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5757a7d-857e-4ed9-bf8d-bb214b1b36be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:43.066356 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5757a7d-857e-4ed9-bf8d-bb214b1b36be admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:43.122789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 765/1565] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:42 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:43.127543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd88c0-2764-4270-a3a3-3e5a91e3bcbe None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:43.143911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd88c0-2764-4270-a3a3-3e5a91e3bcbe None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:43.219786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:43.239767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08c992-77f7-4c5f-9599-97c5d7e72b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:43.241434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08c992-77f7-4c5f-9599-97c5d7e72b07 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:43.242944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08c992-77f7-4c5f-9599-97c5d7e72b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:43.245032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08c992-77f7-4c5f-9599-97c5d7e72b07 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:43.245760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08c992-77f7-4c5f-9599-97c5d7e72b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:43.251076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08c992-77f7-4c5f-9599-97c5d7e72b07 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:43.289586 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 801/1566] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:43 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:43.293782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5757a7d-857e-4ed9-bf8d-bb214b1b36be admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:43.305852 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5757a7d-857e-4ed9-bf8d-bb214b1b36be admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:43.310964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efcaa1-1511-4487-866d-e31f46f13bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:43.311580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efcaa1-1511-4487-866d-e31f46f13bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:43.311908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efcaa1-1511-4487-866d-e31f46f13bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:43.312825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efcaa1-1511-4487-866d-e31f46f13bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:43.312825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efcaa1-1511-4487-866d-e31f46f13bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:43.314395 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efcaa1-1511-4487-866d-e31f46f13bbb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:43.330673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 766/1567] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:55:43.337953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08c992-77f7-4c5f-9599-97c5d7e72b07 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:43.355166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08c992-77f7-4c5f-9599-97c5d7e72b07 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:43.369579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f46c2-e905-4ced-be9b-8850eb48bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:43.370459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f46c2-e905-4ced-be9b-8850eb48bed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:43.370905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f46c2-e905-4ced-be9b-8850eb48bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:43.371845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f46c2-e905-4ced-be9b-8850eb48bed0 admin admin] PATH_INFO: `/v3/projects/70f0bad0d6fc40f48ac8025914ef10e3/users/43789e07db8a4782b918064c8799f31a/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:43.450443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f46c2-e905-4ced-be9b-8850eb48bed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f0bad0d6fc40f48ac8025914ef10e3, user_id=43789e07db8a4782b918064c8799f31a, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:43.454319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f46c2-e905-4ced-be9b-8850eb48bed0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:43.473923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 802/1568] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:43 2022] PUT /identity/v3/projects/70f0bad0d6fc40f48ac8025914ef10e3/users/43789e07db8a4782b918064c8799f31a/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:43.478717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efcaa1-1511-4487-866d-e31f46f13bbb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:43.489007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efcaa1-1511-4487-866d-e31f46f13bbb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:43.503934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:43.544681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f46c2-e905-4ced-be9b-8850eb48bed0 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:43.547601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ea0f46c2-e905-4ced-be9b-8850eb48bed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:43.646685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b096c-6fcc-45ed-9da0-1ac68ea01927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:43.647311 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b096c-6fcc-45ed-9da0-1ac68ea01927 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:43.647589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b096c-6fcc-45ed-9da0-1ac68ea01927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:43.648289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b096c-6fcc-45ed-9da0-1ac68ea01927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:43.649129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b096c-6fcc-45ed-9da0-1ac68ea01927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:43.651504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b096c-6fcc-45ed-9da0-1ac68ea01927 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:43.672842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 767/1569] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:43.673840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b096c-6fcc-45ed-9da0-1ac68ea01927 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:43.676301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3d0b096c-6fcc-45ed-9da0-1ac68ea01927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:43.696808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b096c-6fcc-45ed-9da0-1ac68ea01927 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:43.704964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:43.706244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:43.706517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:43.706794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] PATH_INFO: `/v3/users/b89f68d663164421b4c15e31d0b8623b` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:43.708612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b89f68d663164421b4c15e31d0b8623b)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:43.712270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:43.716822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f46c2-e905-4ced-be9b-8850eb48bed0 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:43.723946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:43.724978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:43.725454 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:43.726070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] PATH_INFO: `/v3/projects/368f4c071772497aa3811eb2dd080a35` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:43.729779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=368f4c071772497aa3811eb2dd080a35)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:43.733503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:43.814150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b89f68d663164421b4c15e31d0b8623b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:43.815008 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b89f68d663164421b4c15e31d0b8623b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:43.847448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b89f68d663164421b4c15e31d0b8623b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:43.850290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b89f68d663164421b4c15e31d0b8623b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:43.864459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '368f4c071772497aa3811eb2dd080a35', 'request_id': 'req-8384dc27-11fd-4fd7-b1a2-d365625b56e8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:43.880303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '368f4c071772497aa3811eb2dd080a35', 'request_id': 'req-8384dc27-11fd-4fd7-b1a2-d365625b56e8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:43.888873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b89f68d663164421b4c15e31d0b8623b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:43.897653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '368f4c071772497aa3811eb2dd080a35', 'request_id': 'req-8384dc27-11fd-4fd7-b1a2-d365625b56e8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:43.921927 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 768/1570] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:43 2022] DELETE /identity/v3/users/b89f68d663164421b4c15e31d0b8623b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:43.923134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:43.925014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:43.954540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] The token cache is being invalidate because project 368f4c071772497aa3811eb2dd080a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:43.954930 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:43.956315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8384dc27-11fd-4fd7-b1a2-d365625b56e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:43.966821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 803/1571] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:43 2022] DELETE /identity/v3/projects/368f4c071772497aa3811eb2dd080a35 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:43.968271 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be9eb2-bb7a-4d30-86f5-699f5ab34378 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:43.968544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be9eb2-bb7a-4d30-86f5-699f5ab34378 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:43.968807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be9eb2-bb7a-4d30-86f5-699f5ab34378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:44.002846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e9be9eb2-bb7a-4d30-86f5-699f5ab34378 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:44.004252 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e9be9eb2-bb7a-4d30-86f5-699f5ab34378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:44.068004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad7eb9-6c86-4325-87ee-3a0d8cc0e967 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:44.074978 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d68f17-12af-4dfa-a823-c03d314e5700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:44.076583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68f17-12af-4dfa-a823-c03d314e5700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:44.076583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68f17-12af-4dfa-a823-c03d314e5700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:44.076583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68f17-12af-4dfa-a823-c03d314e5700 admin admin] PATH_INFO: `/v3/projects/70f0bad0d6fc40f48ac8025914ef10e3/users/43789e07db8a4782b918064c8799f31a/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:44.078632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d68f17-12af-4dfa-a823-c03d314e5700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f0bad0d6fc40f48ac8025914ef10e3, user_id=43789e07db8a4782b918064c8799f31a, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:44.082253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d68f17-12af-4dfa-a823-c03d314e5700 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:44.107441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 769/1572] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:43 2022] PUT /identity/v3/projects/70f0bad0d6fc40f48ac8025914ef10e3/users/43789e07db8a4782b918064c8799f31a/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:44.107441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d68f17-12af-4dfa-a823-c03d314e5700 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:44.107441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d6d68f17-12af-4dfa-a823-c03d314e5700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:44.124078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:44.135804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:44.172500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 804/1573] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:44.175038 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be9eb2-bb7a-4d30-86f5-699f5ab34378 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:44.178698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e9be9eb2-bb7a-4d30-86f5-699f5ab34378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:44.195067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be9eb2-bb7a-4d30-86f5-699f5ab34378 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:44.232348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d68f17-12af-4dfa-a823-c03d314e5700 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:44.239688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:44.242877 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:44.244201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:44.244692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] PATH_INFO: `/v3/users/2160c4cd63294e6683e85932233d8c1f` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:44.250561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2160c4cd63294e6683e85932233d8c1f)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:44.253244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:44.313499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:44.314147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:44.314425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:44.314724 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] PATH_INFO: `/v3/projects/32dab4af21f74138b614f6a0b0a30f13` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:44.316316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32dab4af21f74138b614f6a0b0a30f13)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:44.318250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:44.326374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2160c4cd63294e6683e85932233d8c1f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.328035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2160c4cd63294e6683e85932233d8c1f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.338245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2160c4cd63294e6683e85932233d8c1f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.339210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2160c4cd63294e6683e85932233d8c1f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.358947 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-65cc908b-adb6-47b3-8c0b-dced821c174f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2160c4cd63294e6683e85932233d8c1f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.387766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 770/1574] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:44 2022] DELETE /identity/v3/users/2160c4cd63294e6683e85932233d8c1f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:44.389959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b9400-2374-4dda-9eed-ea5a08e228a7 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:44.391138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b9400-2374-4dda-9eed-ea5a08e228a7 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:44.391623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b9400-2374-4dda-9eed-ea5a08e228a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:44.396449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32dab4af21f74138b614f6a0b0a30f13', 'request_id': 'req-76d610e3-efc0-4990-9517-a07767961361'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.403603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32dab4af21f74138b614f6a0b0a30f13', 'request_id': 'req-76d610e3-efc0-4990-9517-a07767961361'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.417233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32dab4af21f74138b614f6a0b0a30f13', 'request_id': 'req-76d610e3-efc0-4990-9517-a07767961361'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.439971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] The token cache is being invalidate because project 32dab4af21f74138b614f6a0b0a30f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:44.439971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.439971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.439971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 805/1575] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:44 2022] DELETE /identity/v3/projects/32dab4af21f74138b614f6a0b0a30f13 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:44.439971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:44.440833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:44.452817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-8d4b9400-2374-4dda-9eed-ea5a08e228a7 None None] MFA Rules not processed for user `43789e07db8a4782b918064c8799f31a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:44.454288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8d4b9400-2374-4dda-9eed-ea5a08e228a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:44.512822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d610e3-efc0-4990-9517-a07767961361 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:44.566224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:44.613076 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 771/1576] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:44.615251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c22b20-f4f9-4a0c-a460-49548172837b None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:44.615533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c22b20-f4f9-4a0c-a460-49548172837b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:44.615807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c22b20-f4f9-4a0c-a460-49548172837b None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:44.620103 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 772/1577] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:55:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:55:44.621012 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c22b20-f4f9-4a0c-a460-49548172837b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:44.622847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b1c22b20-f4f9-4a0c-a460-49548172837b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:44.629261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef451b-9bab-43dc-8a91-0304f5fa70fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:44.630084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef451b-9bab-43dc-8a91-0304f5fa70fd service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:44.630374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef451b-9bab-43dc-8a91-0304f5fa70fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:44.630703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef451b-9bab-43dc-8a91-0304f5fa70fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:44.633444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-75ef451b-9bab-43dc-8a91-0304f5fa70fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:44.670851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:44.735089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c22b20-f4f9-4a0c-a460-49548172837b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:44.743958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef451b-9bab-43dc-8a91-0304f5fa70fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:44.746403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef451b-9bab-43dc-8a91-0304f5fa70fd service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:44.748382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:44.749066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:44.749347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:44.749621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] PATH_INFO: `/v3/users/19a2aad4a4d843ddbeba43dd8cadc93d` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:44.753374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19a2aad4a4d843ddbeba43dd8cadc93d)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:44.757302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:44.760389 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 806/1578] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:44.762239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef451b-9bab-43dc-8a91-0304f5fa70fd service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:44.763839 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-75ef451b-9bab-43dc-8a91-0304f5fa70fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:44.848157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19a2aad4a4d843ddbeba43dd8cadc93d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.850829 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19a2aad4a4d843ddbeba43dd8cadc93d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.860827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19a2aad4a4d843ddbeba43dd8cadc93d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.862179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19a2aad4a4d843ddbeba43dd8cadc93d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.868530 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef451b-9bab-43dc-8a91-0304f5fa70fd service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:44.876094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59055f8a-cfc6-4fa7-a199-e429de6c18ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:44.876891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59055f8a-cfc6-4fa7-a199-e429de6c18ee service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:44.877247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59055f8a-cfc6-4fa7-a199-e429de6c18ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:44.877592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59055f8a-cfc6-4fa7-a199-e429de6c18ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:44.882675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-59055f8a-cfc6-4fa7-a199-e429de6c18ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:44.884636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-967a4e06-9c3c-400d-a1f0-78c7de2bac69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19a2aad4a4d843ddbeba43dd8cadc93d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:44.903089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59055f8a-cfc6-4fa7-a199-e429de6c18ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:44.905098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59055f8a-cfc6-4fa7-a199-e429de6c18ee service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:44.916205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 773/1579] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:44 2022] DELETE /identity/v3/users/19a2aad4a4d843ddbeba43dd8cadc93d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:44.917875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf05dc5-957a-4e30-857b-54af6417f15a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:44.918302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf05dc5-957a-4e30-857b-54af6417f15a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:44.918677 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf05dc5-957a-4e30-857b-54af6417f15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:44.927368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 807/1580] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:44.943252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2bf05dc5-957a-4e30-857b-54af6417f15a None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:44.944744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2bf05dc5-957a-4e30-857b-54af6417f15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:45.070410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 774/1581] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:55:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:45.132696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59055f8a-cfc6-4fa7-a199-e429de6c18ee service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:45.135648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-59055f8a-cfc6-4fa7-a199-e429de6c18ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:45.333746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59055f8a-cfc6-4fa7-a199-e429de6c18ee service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:45.347540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef6b49-4558-414b-829d-71b8daa7fd56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:45.348418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef6b49-4558-414b-829d-71b8daa7fd56 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:45.348695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef6b49-4558-414b-829d-71b8daa7fd56 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:45.348971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef6b49-4558-414b-829d-71b8daa7fd56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:45.397566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef6b49-4558-414b-829d-71b8daa7fd56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:45.406422 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef6b49-4558-414b-829d-71b8daa7fd56 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:45.447209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 808/1582] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:45.529420 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf05dc5-957a-4e30-857b-54af6417f15a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:45.531232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2bf05dc5-957a-4e30-857b-54af6417f15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:45.639641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:45.735979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf05dc5-957a-4e30-857b-54af6417f15a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:45.743695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:45.744627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:45.745010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:45.745367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] PATH_INFO: `/v3/users/23f77ae09e6e44e1bc68842090d5a328` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:45.747345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f77ae09e6e44e1bc68842090d5a328)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:45.749447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:45.815193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f77ae09e6e44e1bc68842090d5a328'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:45.817662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f77ae09e6e44e1bc68842090d5a328'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:45.830366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f77ae09e6e44e1bc68842090d5a328'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:45.831657 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f77ae09e6e44e1bc68842090d5a328'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:45.856909 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f77ae09e6e44e1bc68842090d5a328'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:45.900639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 775/1583] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:45 2022] DELETE /identity/v3/users/23f77ae09e6e44e1bc68842090d5a328 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:45.905204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eee386-474b-4172-8dd0-07fcefaf736d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:45.905751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eee386-474b-4172-8dd0-07fcefaf736d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:45.905751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eee386-474b-4172-8dd0-07fcefaf736d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:45.929350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-60eee386-474b-4172-8dd0-07fcefaf736d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:45.931692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-60eee386-474b-4172-8dd0-07fcefaf736d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:45.943064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:46.003057 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 809/1584] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:46.033857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:46.036189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:46.103727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2815c-5236-4fc9-8c5f-c788eaebad8a admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:46.109714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f3311-5316-4852-9e60-20299ddbb8c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:46.110985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f3311-5316-4852-9e60-20299ddbb8c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:46.110985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f3311-5316-4852-9e60-20299ddbb8c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:46.111164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f3311-5316-4852-9e60-20299ddbb8c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:46.126057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f3311-5316-4852-9e60-20299ddbb8c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:46.127783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f3311-5316-4852-9e60-20299ddbb8c2 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:46.141338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 776/1585] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:46.245512 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eee386-474b-4172-8dd0-07fcefaf736d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:46.247306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-60eee386-474b-4172-8dd0-07fcefaf736d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:46.281214 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eee386-474b-4172-8dd0-07fcefaf736d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:46.305575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:46.306517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:46.306815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:46.307609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] PATH_INFO: `/v3/projects/f13a487274cf4accac4b65eb022c6201` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:46.313679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f13a487274cf4accac4b65eb022c6201)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:46.316655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:46.357053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f3311-5316-4852-9e60-20299ddbb8c2 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:46.359862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-361f3311-5316-4852-9e60-20299ddbb8c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:46.379486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f3311-5316-4852-9e60-20299ddbb8c2 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:46.400721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:46.400911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:46.401148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:46.401812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] PATH_INFO: `/v3/projects/ee40f2f0dd7e46eaa91cee4a7e26496d` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:46.403041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee40f2f0dd7e46eaa91cee4a7e26496d)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:46.405339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:46.444948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f13a487274cf4accac4b65eb022c6201', 'request_id': 'req-f19ffc25-e5db-4e4e-9043-8633503cf756'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.453798 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f13a487274cf4accac4b65eb022c6201', 'request_id': 'req-f19ffc25-e5db-4e4e-9043-8633503cf756'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.466662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f13a487274cf4accac4b65eb022c6201', 'request_id': 'req-f19ffc25-e5db-4e4e-9043-8633503cf756'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.493734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] The token cache is being invalidate because project f13a487274cf4accac4b65eb022c6201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:46.494247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.495151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f19ffc25-e5db-4e4e-9043-8633503cf756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.497048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 810/1586] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:46 2022] DELETE /identity/v3/projects/f13a487274cf4accac4b65eb022c6201 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:46.501685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969b200-f390-48ba-9aa9-a9cbfcbce2af None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:46.502218 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969b200-f390-48ba-9aa9-a9cbfcbce2af None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:46.502700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969b200-f390-48ba-9aa9-a9cbfcbce2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:46.516086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:46.522586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee40f2f0dd7e46eaa91cee4a7e26496d', 'request_id': 'req-3546eab9-889d-4c64-bf90-928766020573'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.543220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee40f2f0dd7e46eaa91cee4a7e26496d', 'request_id': 'req-3546eab9-889d-4c64-bf90-928766020573'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.549144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-3969b200-f390-48ba-9aa9-a9cbfcbce2af None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:46.551736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3969b200-f390-48ba-9aa9-a9cbfcbce2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:46.561961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee40f2f0dd7e46eaa91cee4a7e26496d', 'request_id': 'req-3546eab9-889d-4c64-bf90-928766020573'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.580871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] The token cache is being invalidate because project ee40f2f0dd7e46eaa91cee4a7e26496d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:46.582860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.584959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.589526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 777/1587] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:46 2022] DELETE /identity/v3/projects/ee40f2f0dd7e46eaa91cee4a7e26496d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:46.594646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:46.598276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:46.640631 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 811/1588] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:46.650190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271f766-8f54-40fe-8fdf-8db56d9b431b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:46.652085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271f766-8f54-40fe-8fdf-8db56d9b431b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:46.652873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271f766-8f54-40fe-8fdf-8db56d9b431b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:46.677391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2271f766-8f54-40fe-8fdf-8db56d9b431b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:46.678764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2271f766-8f54-40fe-8fdf-8db56d9b431b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:46.692987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 812/1589] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:46.694596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108070e-d2d8-4fee-8f76-d2dcd94b2f55 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:46.695010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108070e-d2d8-4fee-8f76-d2dcd94b2f55 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:46.696698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108070e-d2d8-4fee-8f76-d2dcd94b2f55 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:46.698896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 813/1590] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:55:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:55:46.699859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108070e-d2d8-4fee-8f76-d2dcd94b2f55 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:46.716530 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546eab9-889d-4c64-bf90-928766020573 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:46.719488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108070e-d2d8-4fee-8f76-d2dcd94b2f55 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:46.722668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:46.723761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:46.724146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:46.724362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] PATH_INFO: `/v3/users/6187de6906e54eed9419fa9203543f64` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:46.725783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6187de6906e54eed9419fa9203543f64)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:46.728344 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:46.731808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab5068-dd9a-4c81-be94-12b51d795b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:46.733188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab5068-dd9a-4c81-be94-12b51d795b97 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:46.733341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab5068-dd9a-4c81-be94-12b51d795b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:46.733841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab5068-dd9a-4c81-be94-12b51d795b97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:46.735106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab5068-dd9a-4c81-be94-12b51d795b97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:46.735471 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab5068-dd9a-4c81-be94-12b51d795b97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:46.737454 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab5068-dd9a-4c81-be94-12b51d795b97 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:46.770230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 814/1591] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:46.770230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd558a-316e-4bbe-9601-010a3426fc8b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:46.770230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd558a-316e-4bbe-9601-010a3426fc8b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:46.770230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd558a-316e-4bbe-9601-010a3426fc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:46.810818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-cccd558a-316e-4bbe-9601-010a3426fc8b None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:46.811590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cccd558a-316e-4bbe-9601-010a3426fc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:46.831807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:46.834989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6187de6906e54eed9419fa9203543f64'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.837043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6187de6906e54eed9419fa9203543f64'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.850965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6187de6906e54eed9419fa9203543f64'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.852567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6187de6906e54eed9419fa9203543f64'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.866871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6187de6906e54eed9419fa9203543f64'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:46.886197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 778/1592] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:46 2022] DELETE /identity/v3/users/6187de6906e54eed9419fa9203543f64 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:46.887130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:46.888626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:46.915388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 815/1593] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:55:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:46.916212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd558a-316e-4bbe-9601-010a3426fc8b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:46.917819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cccd558a-316e-4bbe-9601-010a3426fc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:46.931578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd558a-316e-4bbe-9601-010a3426fc8b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:46.972974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:46.983933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115935f0-96ea-46d5-9f0d-f55153c8710b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:46.994907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:46.995134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:46.997248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:46.997707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] PATH_INFO: `/v3/projects/9b4cf02e97f9472cb5075dee9a1baa14` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:46.999273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4cf02e97f9472cb5075dee9a1baa14)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:47.002419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:47.026619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8f1a1-bba7-410f-b1d9-558e081c4a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:47.027420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8f1a1-bba7-410f-b1d9-558e081c4a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:47.027916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8f1a1-bba7-410f-b1d9-558e081c4a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:47.028136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8f1a1-bba7-410f-b1d9-558e081c4a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:47.028959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8f1a1-bba7-410f-b1d9-558e081c4a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:47.031225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8f1a1-bba7-410f-b1d9-558e081c4a3c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:47.085098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 816/1594] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:46 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:47.086692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8f1a1-bba7-410f-b1d9-558e081c4a3c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:47.089182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d1f8f1a1-bba7-410f-b1d9-558e081c4a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.103368 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4cf02e97f9472cb5075dee9a1baa14', 'request_id': 'req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.116982 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4cf02e97f9472cb5075dee9a1baa14', 'request_id': 'req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.137245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4cf02e97f9472cb5075dee9a1baa14', 'request_id': 'req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.162472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] The token cache is being invalidate because project 9b4cf02e97f9472cb5075dee9a1baa14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:47.163127 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.166225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.171263 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 779/1595] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:46 2022] DELETE /identity/v3/projects/9b4cf02e97f9472cb5075dee9a1baa14 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:47.172393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:47.176533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.210073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8f1a1-bba7-410f-b1d9-558e081c4a3c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:47.227871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:47.228613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:47.228885 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:47.229150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] PATH_INFO: `/v3/projects/144ac59f1f4b4f17ac96f5406cfe9d5e` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:47.232634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=144ac59f1f4b4f17ac96f5406cfe9d5e)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:47.234980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:47.284812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac01aa-1937-4c60-ab29-3f0b566ea9f2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:47.294828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:47.311799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '144ac59f1f4b4f17ac96f5406cfe9d5e', 'request_id': 'req-a8bc234b-151d-4df2-a27b-d390192ff0f5'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.322136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '144ac59f1f4b4f17ac96f5406cfe9d5e', 'request_id': 'req-a8bc234b-151d-4df2-a27b-d390192ff0f5'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.333644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '144ac59f1f4b4f17ac96f5406cfe9d5e', 'request_id': 'req-a8bc234b-151d-4df2-a27b-d390192ff0f5'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.361766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] The token cache is being invalidate because project 144ac59f1f4b4f17ac96f5406cfe9d5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:47.362718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.363544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.366007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 817/1596] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:47 2022] DELETE /identity/v3/projects/144ac59f1f4b4f17ac96f5406cfe9d5e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:47.367510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:47.377820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.395129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b979fb-6a16-472e-af7a-9c3ad18fd40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:47.395923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b979fb-6a16-472e-af7a-9c3ad18fd40b admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:47.396641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b979fb-6a16-472e-af7a-9c3ad18fd40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:47.397017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b979fb-6a16-472e-af7a-9c3ad18fd40b admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:47.397719 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b979fb-6a16-472e-af7a-9c3ad18fd40b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:47.400451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b979fb-6a16-472e-af7a-9c3ad18fd40b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:47.440924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 780/1597] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:47 2022] POST /identity/v3/users => generated 292 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:47.442423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af877e63-035d-4715-bc7d-e5b1f7c97c9a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:47.442706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af877e63-035d-4715-bc7d-e5b1f7c97c9a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:47.443030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af877e63-035d-4715-bc7d-e5b1f7c97c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:47.474495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-af877e63-035d-4715-bc7d-e5b1f7c97c9a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:47.475870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-af877e63-035d-4715-bc7d-e5b1f7c97c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.493092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 781/1598] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:47.493092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af877e63-035d-4715-bc7d-e5b1f7c97c9a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:47.495564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-af877e63-035d-4715-bc7d-e5b1f7c97c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.498435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc234b-151d-4df2-a27b-d390192ff0f5 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:47.507629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a5399-3309-43ca-9cdb-4c1d0b273072 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:47.508867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a5399-3309-43ca-9cdb-4c1d0b273072 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:47.509246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a5399-3309-43ca-9cdb-4c1d0b273072 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:47.509521 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a5399-3309-43ca-9cdb-4c1d0b273072 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:47.511161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af877e63-035d-4715-bc7d-e5b1f7c97c9a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:47.512471 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-523a5399-3309-43ca-9cdb-4c1d0b273072 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.516654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:47.517313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:47.517585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:47.517853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] PATH_INFO: `/v3/projects/f671b65573b245559a27eca54dbe4dcb` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:47.519094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f671b65573b245559a27eca54dbe4dcb)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:47.521400 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:47.534745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a5399-3309-43ca-9cdb-4c1d0b273072 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:47.536406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a5399-3309-43ca-9cdb-4c1d0b273072 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:47.537205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:47.560226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 818/1599] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:47.561241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a5399-3309-43ca-9cdb-4c1d0b273072 service nova] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:47.564486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-523a5399-3309-43ca-9cdb-4c1d0b273072 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.584854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a5399-3309-43ca-9cdb-4c1d0b273072 service nova] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:47.587227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f671b65573b245559a27eca54dbe4dcb', 'request_id': 'req-8498adbe-646c-4d83-8568-d1997bdb5967'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.594373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f671b65573b245559a27eca54dbe4dcb', 'request_id': 'req-8498adbe-646c-4d83-8568-d1997bdb5967'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.612318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f671b65573b245559a27eca54dbe4dcb', 'request_id': 'req-8498adbe-646c-4d83-8568-d1997bdb5967'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.633550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] The token cache is being invalidate because project f671b65573b245559a27eca54dbe4dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:47.633842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.634810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.641324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 782/1600] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:47 2022] DELETE /identity/v3/projects/f671b65573b245559a27eca54dbe4dcb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:47.641324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:47.641324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.677258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:47.677903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:47.678278 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:47.678552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] PATH_INFO: `/v3/users/43789e07db8a4782b918064c8799f31a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:47.684031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43789e07db8a4782b918064c8799f31a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:47.686844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:47.736005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498adbe-646c-4d83-8568-d1997bdb5967 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:47.745147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52ef03-e482-4662-a488-ac096c7c6388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:47.745914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ef03-e482-4662-a488-ac096c7c6388 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:47.746281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ef03-e482-4662-a488-ac096c7c6388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:47.746590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ef03-e482-4662-a488-ac096c7c6388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:47.747315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52ef03-e482-4662-a488-ac096c7c6388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:47.749042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52ef03-e482-4662-a488-ac096c7c6388 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:47.760038 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 783/1601] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:47.764455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fdeaa-67ef-4ee2-9cad-d339ec294a41 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:47.764809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fdeaa-67ef-4ee2-9cad-d339ec294a41 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:47.765158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fdeaa-67ef-4ee2-9cad-d339ec294a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:47.772721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43789e07db8a4782b918064c8799f31a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.775920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43789e07db8a4782b918064c8799f31a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.783785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:47.788280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43789e07db8a4782b918064c8799f31a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.788689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-211fdeaa-67ef-4ee2-9cad-d339ec294a41 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:47.790875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-211fdeaa-67ef-4ee2-9cad-d339ec294a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.795521 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43789e07db8a4782b918064c8799f31a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.813314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 784/1602] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:47.814945 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2a971-dd1b-4a3f-b494-e8280d9aebe8 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:47.815285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2a971-dd1b-4a3f-b494-e8280d9aebe8 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:47.815716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2a971-dd1b-4a3f-b494-e8280d9aebe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:47.828658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43789e07db8a4782b918064c8799f31a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:47.839789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f4e2a971-dd1b-4a3f-b494-e8280d9aebe8 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:47.841085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f4e2a971-dd1b-4a3f-b494-e8280d9aebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.858483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 785/1603] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:47.858483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2a971-dd1b-4a3f-b494-e8280d9aebe8 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:47.858483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f4e2a971-dd1b-4a3f-b494-e8280d9aebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:47.873345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 819/1604] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:47 2022] DELETE /identity/v3/users/43789e07db8a4782b918064c8799f31a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:47.873966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:47.880647 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2a971-dd1b-4a3f-b494-e8280d9aebe8 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.017422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31859411-f872-4037-b560-ede1dd77f79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.018006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31859411-f872-4037-b560-ede1dd77f79d admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.018337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31859411-f872-4037-b560-ede1dd77f79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.018617 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31859411-f872-4037-b560-ede1dd77f79d admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.019429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31859411-f872-4037-b560-ede1dd77f79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.022428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31859411-f872-4037-b560-ede1dd77f79d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.043277 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 786/1605] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:48.044738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f9dd8-f3fb-4f25-8afc-3ce5145b0bd3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.045468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f9dd8-f3fb-4f25-8afc-3ce5145b0bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.045468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f9dd8-f3fb-4f25-8afc-3ce5145b0bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.066770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24afbfd-1292-4adf-8671-7333981e5282 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.071559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-344f9dd8-f3fb-4f25-8afc-3ce5145b0bd3 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:48.073947 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-344f9dd8-f3fb-4f25-8afc-3ce5145b0bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:48.074971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c2c91-8b78-4ed7-bea9-8cc9de880f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.076317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c2c91-8b78-4ed7-bea9-8cc9de880f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.077425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c2c91-8b78-4ed7-bea9-8cc9de880f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.078385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c2c91-8b78-4ed7-bea9-8cc9de880f61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.081121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c2c91-8b78-4ed7-bea9-8cc9de880f61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:48.081649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c2c91-8b78-4ed7-bea9-8cc9de880f61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.083731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c2c91-8b78-4ed7-bea9-8cc9de880f61 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.088911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 787/1606] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:48.088911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f9dd8-f3fb-4f25-8afc-3ce5145b0bd3 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.099783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f9dd8-f3fb-4f25-8afc-3ce5145b0bd3 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.105604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 820/1607] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:48.106856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c2c91-8b78-4ed7-bea9-8cc9de880f61 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.110782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93218d34-20e6-4921-b983-10d40d02d9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.111734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93218d34-20e6-4921-b983-10d40d02d9da admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.112332 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93218d34-20e6-4921-b983-10d40d02d9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.112432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93218d34-20e6-4921-b983-10d40d02d9da admin admin] PATH_INFO: `/v3/projects/7088c0a98f0f40b8b1704fd601ae8810/users/89097ec7688b47e19c16a6b3dc4b2765/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.127747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:48.139792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93218d34-20e6-4921-b983-10d40d02d9da admin admin] RBAC: Authorizing `identity:create_grant(project_id=7088c0a98f0f40b8b1704fd601ae8810, user_id=89097ec7688b47e19c16a6b3dc4b2765, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.144786 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93218d34-20e6-4921-b983-10d40d02d9da admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.152638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c2c91-8b78-4ed7-bea9-8cc9de880f61 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.169819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7daa8-63b7-4073-94fc-743cfa344d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.171781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7daa8-63b7-4073-94fc-743cfa344d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.172912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7daa8-63b7-4073-94fc-743cfa344d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.173201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7daa8-63b7-4073-94fc-743cfa344d7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.174350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7daa8-63b7-4073-94fc-743cfa344d7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:48.174712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7daa8-63b7-4073-94fc-743cfa344d7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.177881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7daa8-63b7-4073-94fc-743cfa344d7f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.195009 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 788/1608] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:48 2022] PUT /identity/v3/projects/7088c0a98f0f40b8b1704fd601ae8810/users/89097ec7688b47e19c16a6b3dc4b2765/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:48.207336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93218d34-20e6-4921-b983-10d40d02d9da admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.216402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 821/1609] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:48.220449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c371-9f60-4453-baf9-c1739cd54f0e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.222540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c371-9f60-4453-baf9-c1739cd54f0e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.223260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c371-9f60-4453-baf9-c1739cd54f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.310237 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0e40c371-9f60-4453-baf9-c1739cd54f0e None None] MFA Rules not processed for user `89097ec7688b47e19c16a6b3dc4b2765`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:48.312381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0e40c371-9f60-4453-baf9-c1739cd54f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:48.350209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93218d34-20e6-4921-b983-10d40d02d9da admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.358303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20470d-d793-42b9-91dc-df3e5cf4790a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.359170 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20470d-d793-42b9-91dc-df3e5cf4790a admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.359687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20470d-d793-42b9-91dc-df3e5cf4790a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.360258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20470d-d793-42b9-91dc-df3e5cf4790a admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.361318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20470d-d793-42b9-91dc-df3e5cf4790a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.363742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20470d-d793-42b9-91dc-df3e5cf4790a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.423103 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 789/1610] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:48 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:48.424202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20470d-d793-42b9-91dc-df3e5cf4790a admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.440630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 822/1611] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:48.442416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c371-9f60-4453-baf9-c1739cd54f0e None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.462511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c371-9f60-4453-baf9-c1739cd54f0e None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.520019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:48.553807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52d596-ccac-4972-b10e-33f018e3e3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.555438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52d596-ccac-4972-b10e-33f018e3e3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.555836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52d596-ccac-4972-b10e-33f018e3e3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.556321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52d596-ccac-4972-b10e-33f018e3e3c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.557084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52d596-ccac-4972-b10e-33f018e3e3c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.561199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52d596-ccac-4972-b10e-33f018e3e3c1 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.580843 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20470d-d793-42b9-91dc-df3e5cf4790a admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.595549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 823/1612] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:48 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:48.597322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52d596-ccac-4972-b10e-33f018e3e3c1 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.599067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c4a96-70e8-45a9-b547-5afddb17c0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.600082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c4a96-70e8-45a9-b547-5afddb17c0eb admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.600367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c4a96-70e8-45a9-b547-5afddb17c0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.600638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c4a96-70e8-45a9-b547-5afddb17c0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.601286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c4a96-70e8-45a9-b547-5afddb17c0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.603256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c4a96-70e8-45a9-b547-5afddb17c0eb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.611414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52d596-ccac-4972-b10e-33f018e3e3c1 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.616970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93ac89-0af6-4384-8a3f-282da4e5c2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.618878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93ac89-0af6-4384-8a3f-282da4e5c2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.619521 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93ac89-0af6-4384-8a3f-282da4e5c2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.619969 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93ac89-0af6-4384-8a3f-282da4e5c2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.620778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93ac89-0af6-4384-8a3f-282da4e5c2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.625012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93ac89-0af6-4384-8a3f-282da4e5c2b8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.652543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 790/1613] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:48 2022] POST /identity/v3/projects => generated 373 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:48.655942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c4a96-70e8-45a9-b547-5afddb17c0eb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.676946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 824/1614] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:48 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:48.677797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93ac89-0af6-4384-8a3f-282da4e5c2b8 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.794672 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:48.797906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93ac89-0af6-4384-8a3f-282da4e5c2b8 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.803131 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c4a96-70e8-45a9-b547-5afddb17c0eb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.807099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321f703-32fb-46a4-9d77-a2a046d268f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.807835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321f703-32fb-46a4-9d77-a2a046d268f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.808313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321f703-32fb-46a4-9d77-a2a046d268f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.808994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321f703-32fb-46a4-9d77-a2a046d268f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.810022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321f703-32fb-46a4-9d77-a2a046d268f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.812449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321f703-32fb-46a4-9d77-a2a046d268f8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.817957 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d0a72-aab7-4730-be7b-d446f289bfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.819315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d0a72-aab7-4730-be7b-d446f289bfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.819603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d0a72-aab7-4730-be7b-d446f289bfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.819906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d0a72-aab7-4730-be7b-d446f289bfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.821664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d0a72-aab7-4730-be7b-d446f289bfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.825850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d0a72-aab7-4730-be7b-d446f289bfd0 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.848474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 791/1615] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:48.849436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d0a72-aab7-4730-be7b-d446f289bfd0 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.849815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 825/1616] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:48 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:48.853677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321f703-32fb-46a4-9d77-a2a046d268f8 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.876778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d0a72-aab7-4730-be7b-d446f289bfd0 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.879110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321f703-32fb-46a4-9d77-a2a046d268f8 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.886000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c2da1-dc4d-49fa-944d-45c92c8e6488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.886865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c2da1-dc4d-49fa-944d-45c92c8e6488 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.887241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c2da1-dc4d-49fa-944d-45c92c8e6488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.887623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c2da1-dc4d-49fa-944d-45c92c8e6488 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.888635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c2da1-dc4d-49fa-944d-45c92c8e6488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.890959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7c8f9-2525-466f-947a-39eff68debca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.891696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c2da1-dc4d-49fa-944d-45c92c8e6488 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.892010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7c8f9-2525-466f-947a-39eff68debca admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.892427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7c8f9-2525-466f-947a-39eff68debca admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.892837 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7c8f9-2525-466f-947a-39eff68debca admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.893620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7c8f9-2525-466f-947a-39eff68debca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.896110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7c8f9-2525-466f-947a-39eff68debca admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.916864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 826/1617] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:48.918771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7c8f9-2525-466f-947a-39eff68debca admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.927102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 792/1618] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:48 2022] POST /identity/v3/users => generated 298 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:48.928084 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c2da1-dc4d-49fa-944d-45c92c8e6488 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.934316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7c8f9-2525-466f-947a-39eff68debca admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.940745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d811397-6222-4dd1-a5d8-eea321962f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.941493 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d811397-6222-4dd1-a5d8-eea321962f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.941632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d811397-6222-4dd1-a5d8-eea321962f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.941908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d811397-6222-4dd1-a5d8-eea321962f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.943152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d811397-6222-4dd1-a5d8-eea321962f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:48.945986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d811397-6222-4dd1-a5d8-eea321962f98 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:48.953802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c2da1-dc4d-49fa-944d-45c92c8e6488 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:48.964654 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 827/1619] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:48.969520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d811397-6222-4dd1-a5d8-eea321962f98 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:48.970315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651da72-f8b8-4363-8157-70e9cc63df2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:48.971171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651da72-f8b8-4363-8157-70e9cc63df2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:48.971572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651da72-f8b8-4363-8157-70e9cc63df2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:48.972972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651da72-f8b8-4363-8157-70e9cc63df2b admin admin] PATH_INFO: `/v3/projects/813d1003a59e4dad9aa7764c2f1903ad/users/7706eb2365ab4a7f8b28711feae3c22f/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:48.983289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d811397-6222-4dd1-a5d8-eea321962f98 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:49.003740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a934d-6436-4d86-9bdc-5cc8154f038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:49.004955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a934d-6436-4d86-9bdc-5cc8154f038c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.006471 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a934d-6436-4d86-9bdc-5cc8154f038c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.006759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a934d-6436-4d86-9bdc-5cc8154f038c admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.007555 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a934d-6436-4d86-9bdc-5cc8154f038c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:49.014935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651da72-f8b8-4363-8157-70e9cc63df2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=813d1003a59e4dad9aa7764c2f1903ad, user_id=7706eb2365ab4a7f8b28711feae3c22f, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:49.015954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a934d-6436-4d86-9bdc-5cc8154f038c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:49.018341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651da72-f8b8-4363-8157-70e9cc63df2b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:49.050831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 828/1620] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:49.051207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a934d-6436-4d86-9bdc-5cc8154f038c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:49.054716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 793/1621] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:48 2022] PUT /identity/v3/projects/813d1003a59e4dad9aa7764c2f1903ad/users/7706eb2365ab4a7f8b28711feae3c22f/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:49.056756 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651da72-f8b8-4363-8157-70e9cc63df2b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:49.066879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a934d-6436-4d86-9bdc-5cc8154f038c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:49.145754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7c3a5-2e46-4b5a-a01e-acdc0d9f2045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:49.146923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7c3a5-2e46-4b5a-a01e-acdc0d9f2045 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.147392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7c3a5-2e46-4b5a-a01e-acdc0d9f2045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.147867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7c3a5-2e46-4b5a-a01e-acdc0d9f2045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.149484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7c3a5-2e46-4b5a-a01e-acdc0d9f2045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:49.152116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7c3a5-2e46-4b5a-a01e-acdc0d9f2045 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:49.162044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:49.169921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651da72-f8b8-4363-8157-70e9cc63df2b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:49.175719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 829/1622] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:49.177406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b208-37d9-4273-a6cc-f8f6365af9fc None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.179464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222c857-15e9-424d-8305-dbf716e8cb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:49.180561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b208-37d9-4273-a6cc-f8f6365af9fc None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.180844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b208-37d9-4273-a6cc-f8f6365af9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.181265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222c857-15e9-424d-8305-dbf716e8cb0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.181613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222c857-15e9-424d-8305-dbf716e8cb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.181984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222c857-15e9-424d-8305-dbf716e8cb0c admin admin] PATH_INFO: `/v3/projects/ba717b1f44c542ed8b261f2b8455c57a/users/0ed4a92fe9ef415eba52f4cea63556ce/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.203137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222c857-15e9-424d-8305-dbf716e8cb0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba717b1f44c542ed8b261f2b8455c57a, user_id=0ed4a92fe9ef415eba52f4cea63556ce, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:49.206725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222c857-15e9-424d-8305-dbf716e8cb0c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:49.207828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9e86b208-37d9-4273-a6cc-f8f6365af9fc None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:49.210118 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9e86b208-37d9-4273-a6cc-f8f6365af9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:49.223122 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 830/1623] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:49.224786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86b208-37d9-4273-a6cc-f8f6365af9fc None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:49.230176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 794/1624] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:49 2022] PUT /identity/v3/projects/ba717b1f44c542ed8b261f2b8455c57a/users/0ed4a92fe9ef415eba52f4cea63556ce/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:49.234154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222c857-15e9-424d-8305-dbf716e8cb0c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:49.246679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86b208-37d9-4273-a6cc-f8f6365af9fc None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:49.317466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222c857-15e9-424d-8305-dbf716e8cb0c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:49.327788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95db89-01b4-42c9-9774-578495dae25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:49.328639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95db89-01b4-42c9-9774-578495dae25e admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.329055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95db89-01b4-42c9-9774-578495dae25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.329458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95db89-01b4-42c9-9774-578495dae25e admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.330256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95db89-01b4-42c9-9774-578495dae25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:49.332287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95db89-01b4-42c9-9774-578495dae25e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:49.347128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 795/1625] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:49.349457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95db89-01b4-42c9-9774-578495dae25e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:49.353211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ac95db89-01b4-42c9-9774-578495dae25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:49.370343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b4ed-60e2-49ee-9c6b-fc3913ef141b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:49.371387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b4ed-60e2-49ee-9c6b-fc3913ef141b admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.371805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b4ed-60e2-49ee-9c6b-fc3913ef141b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.372267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b4ed-60e2-49ee-9c6b-fc3913ef141b admin admin] PATH_INFO: `/v3/projects/640684dd19c64642a6984eb8eafa64b3/users/291bf620737e4ddfad7289876f7a6c50/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.409455 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934b4ed-60e2-49ee-9c6b-fc3913ef141b admin admin] RBAC: Authorizing `identity:create_grant(project_id=640684dd19c64642a6984eb8eafa64b3, user_id=291bf620737e4ddfad7289876f7a6c50, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:49.421345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934b4ed-60e2-49ee-9c6b-fc3913ef141b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:49.433360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:49.460247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 831/1626] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:49 2022] PUT /identity/v3/projects/640684dd19c64642a6984eb8eafa64b3/users/291bf620737e4ddfad7289876f7a6c50/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:49.462035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b4ed-60e2-49ee-9c6b-fc3913ef141b admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:49.485662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95db89-01b4-42c9-9774-578495dae25e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:49.596164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b4ed-60e2-49ee-9c6b-fc3913ef141b admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:49.602287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062a76a-9ecb-41cf-8f4e-be23506f753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:49.603081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062a76a-9ecb-41cf-8f4e-be23506f753e admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.603358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062a76a-9ecb-41cf-8f4e-be23506f753e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.603778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062a76a-9ecb-41cf-8f4e-be23506f753e admin admin] PATH_INFO: `/v3/projects/ba717b1f44c542ed8b261f2b8455c57a/users/0ed4a92fe9ef415eba52f4cea63556ce/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.605877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062a76a-9ecb-41cf-8f4e-be23506f753e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba717b1f44c542ed8b261f2b8455c57a, user_id=0ed4a92fe9ef415eba52f4cea63556ce, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:49.609061 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775290a-cc99-4991-8c66-4dd9a8b22ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:49.609890 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775290a-cc99-4991-8c66-4dd9a8b22ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.610034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062a76a-9ecb-41cf-8f4e-be23506f753e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:49.610444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775290a-cc99-4991-8c66-4dd9a8b22ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.610862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775290a-cc99-4991-8c66-4dd9a8b22ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.627082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 832/1627] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:49 2022] PUT /identity/v3/projects/ba717b1f44c542ed8b261f2b8455c57a/users/0ed4a92fe9ef415eba52f4cea63556ce/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:49.629856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ed67-e87f-4381-8a51-46adee60c1f7 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.630136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ed67-e87f-4381-8a51-46adee60c1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.630405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ed67-e87f-4381-8a51-46adee60c1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.630963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775290a-cc99-4991-8c66-4dd9a8b22ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:49.634694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775290a-cc99-4991-8c66-4dd9a8b22ba5 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:49.647696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 796/1628] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:49.649437 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775290a-cc99-4991-8c66-4dd9a8b22ba5 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:49.651193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6775290a-cc99-4991-8c66-4dd9a8b22ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:49.665887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e840ed67-e87f-4381-8a51-46adee60c1f7 None None] MFA Rules not processed for user `291bf620737e4ddfad7289876f7a6c50`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:49.667225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e840ed67-e87f-4381-8a51-46adee60c1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:49.673806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775290a-cc99-4991-8c66-4dd9a8b22ba5 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:49.744509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:49.750679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 833/1629] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:49.752451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5461eeb-1fcc-416d-97ba-0d627eadce0d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.752841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5461eeb-1fcc-416d-97ba-0d627eadce0d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.753183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5461eeb-1fcc-416d-97ba-0d627eadce0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.772318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:49.773359 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:49.773816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:49.774171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] PATH_INFO: `/v3/projects/70f0bad0d6fc40f48ac8025914ef10e3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:49.776779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f0bad0d6fc40f48ac8025914ef10e3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:49.778997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:49.791627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-d5461eeb-1fcc-416d-97ba-0d627eadce0d None None] MFA Rules not processed for user `0ed4a92fe9ef415eba52f4cea63556ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:49.793384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d5461eeb-1fcc-416d-97ba-0d627eadce0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:49.899630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f0bad0d6fc40f48ac8025914ef10e3', 'request_id': 'req-12560bc7-21ee-40e3-be76-ee903fb70fa3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:49.910296 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 834/1630] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:49.910440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f0bad0d6fc40f48ac8025914ef10e3', 'request_id': 'req-12560bc7-21ee-40e3-be76-ee903fb70fa3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:49.911353 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5461eeb-1fcc-416d-97ba-0d627eadce0d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:49.915542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d5461eeb-1fcc-416d-97ba-0d627eadce0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:49.922693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f0bad0d6fc40f48ac8025914ef10e3', 'request_id': 'req-12560bc7-21ee-40e3-be76-ee903fb70fa3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:49.943304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] The token cache is being invalidate because project 70f0bad0d6fc40f48ac8025914ef10e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:49.944342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:49.947301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:49.955714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 797/1631] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:49 2022] DELETE /identity/v3/projects/70f0bad0d6fc40f48ac8025914ef10e3 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:49.958547 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:49.960562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:50.020674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:50.072124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5461eeb-1fcc-416d-97ba-0d627eadce0d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:50.109217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12560bc7-21ee-40e3-be76-ee903fb70fa3 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:50.132328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de98d6-a3c7-4fd1-b6bf-b9d4d2e2da92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:50.133715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de98d6-a3c7-4fd1-b6bf-b9d4d2e2da92 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:50.134259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de98d6-a3c7-4fd1-b6bf-b9d4d2e2da92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:50.135186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de98d6-a3c7-4fd1-b6bf-b9d4d2e2da92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:50.141919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-84de98d6-a3c7-4fd1-b6bf-b9d4d2e2da92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:50.209947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc5897-2993-43d3-8a68-37393d0d96d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:50.210578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc5897-2993-43d3-8a68-37393d0d96d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:50.210851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc5897-2993-43d3-8a68-37393d0d96d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:50.211343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc5897-2993-43d3-8a68-37393d0d96d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:50.215702 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ccbc5897-2993-43d3-8a68-37393d0d96d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:50.319091 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de98d6-a3c7-4fd1-b6bf-b9d4d2e2da92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:50.328055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:50.331752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de98d6-a3c7-4fd1-b6bf-b9d4d2e2da92 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:50.332328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc5897-2993-43d3-8a68-37393d0d96d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:50.334349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc5897-2993-43d3-8a68-37393d0d96d2 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:50.347086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 835/1632] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:50.348687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194d31c-6cb4-4e84-9178-de282e345188 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:50.349285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194d31c-6cb4-4e84-9178-de282e345188 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:50.349693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194d31c-6cb4-4e84-9178-de282e345188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:50.370639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 798/1633] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:50.373843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-b194d31c-6cb4-4e84-9178-de282e345188 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:50.375295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b194d31c-6cb4-4e84-9178-de282e345188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:50.506025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 836/1634] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:50.519746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79489b-f0a1-4f72-a928-c327cd8ef9b3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:50.520027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79489b-f0a1-4f72-a928-c327cd8ef9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:50.520306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79489b-f0a1-4f72-a928-c327cd8ef9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:50.567041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-0d79489b-f0a1-4f72-a928-c327cd8ef9b3 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:50.567041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0d79489b-f0a1-4f72-a928-c327cd8ef9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:50.589972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194d31c-6cb4-4e84-9178-de282e345188 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:50.606116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 799/1635] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:50.608229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194d31c-6cb4-4e84-9178-de282e345188 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:50.610705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79489b-f0a1-4f72-a928-c327cd8ef9b3 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:50.623522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2e71-e7ab-4fec-93bc-5a9b4a5f0e7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:50.624208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2e71-e7ab-4fec-93bc-5a9b4a5f0e7b service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:50.624485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2e71-e7ab-4fec-93bc-5a9b4a5f0e7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:50.624754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2e71-e7ab-4fec-93bc-5a9b4a5f0e7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:50.627960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-714f2e71-e7ab-4fec-93bc-5a9b4a5f0e7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:50.634236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79489b-f0a1-4f72-a928-c327cd8ef9b3 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:50.648274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c0b42-49b2-4c85-9e6b-0d41c47c12fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:50.649901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c0b42-49b2-4c85-9e6b-0d41c47c12fd admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:50.649901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c0b42-49b2-4c85-9e6b-0d41c47c12fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:50.650297 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c0b42-49b2-4c85-9e6b-0d41c47c12fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:50.656559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c0b42-49b2-4c85-9e6b-0d41c47c12fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:50.659449 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c0b42-49b2-4c85-9e6b-0d41c47c12fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:50.663628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c0b42-49b2-4c85-9e6b-0d41c47c12fd admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:50.691537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 800/1636] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:50.696123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c0b42-49b2-4c85-9e6b-0d41c47c12fd admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:50.724597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c0b42-49b2-4c85-9e6b-0d41c47c12fd admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:50.745232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258ea51-3c2d-46e7-a9c4-fd0031506c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:50.750618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258ea51-3c2d-46e7-a9c4-fd0031506c88 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:50.750826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258ea51-3c2d-46e7-a9c4-fd0031506c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:50.751242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258ea51-3c2d-46e7-a9c4-fd0031506c88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:50.752054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258ea51-3c2d-46e7-a9c4-fd0031506c88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:50.755455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258ea51-3c2d-46e7-a9c4-fd0031506c88 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:50.806180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2e71-e7ab-4fec-93bc-5a9b4a5f0e7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:50.807932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2e71-e7ab-4fec-93bc-5a9b4a5f0e7b service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:50.828288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 801/1637] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:50 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:50.829864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258ea51-3c2d-46e7-a9c4-fd0031506c88 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:50.832674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 837/1638] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:50.848021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8f7f0-88a0-4bd7-a990-9e091face0d1 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:50.848661 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8f7f0-88a0-4bd7-a990-9e091face0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:50.849078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8f7f0-88a0-4bd7-a990-9e091face0d1 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:50.850847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 838/1639] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:55:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 17:55:50.936193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42bcd-a936-46c9-9b62-913208ec9504 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:50.936476 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42bcd-a936-46c9-9b62-913208ec9504 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:50.936748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42bcd-a936-46c9-9b62-913208ec9504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.004528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-28b42bcd-a936-46c9-9b62-913208ec9504 None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:51.007732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-28b42bcd-a936-46c9-9b62-913208ec9504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:51.070243 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258ea51-3c2d-46e7-a9c4-fd0031506c88 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:51.095553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c77ba4-9acb-47db-8232-c631524664eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:51.095553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c77ba4-9acb-47db-8232-c631524664eb admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.095844 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c77ba4-9acb-47db-8232-c631524664eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.096001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c77ba4-9acb-47db-8232-c631524664eb admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.096911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c77ba4-9acb-47db-8232-c631524664eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:51.103796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c77ba4-9acb-47db-8232-c631524664eb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:51.163002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 802/1640] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:50 2022] POST /identity/v3/users => generated 291 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:51.169598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c77ba4-9acb-47db-8232-c631524664eb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:51.183507 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c77ba4-9acb-47db-8232-c631524664eb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:51.188844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 839/1641] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:55:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:51.191036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28063e45-a32b-402d-8510-21c5b7901e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:51.191656 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28063e45-a32b-402d-8510-21c5b7901e52 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.191977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28063e45-a32b-402d-8510-21c5b7901e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.192682 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28063e45-a32b-402d-8510-21c5b7901e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.192948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28063e45-a32b-402d-8510-21c5b7901e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:51.195996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28063e45-a32b-402d-8510-21c5b7901e52 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:51.210653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 803/1642] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:51.215552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b42bcd-a936-46c9-9b62-913208ec9504 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:51.230712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b42bcd-a936-46c9-9b62-913208ec9504 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:51.237797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a35ce1-d261-4a5f-9181-b8b2cf7493ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:51.238290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a35ce1-d261-4a5f-9181-b8b2cf7493ae admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.238712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a35ce1-d261-4a5f-9181-b8b2cf7493ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.239078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a35ce1-d261-4a5f-9181-b8b2cf7493ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.240804 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a35ce1-d261-4a5f-9181-b8b2cf7493ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:51.242028 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a35ce1-d261-4a5f-9181-b8b2cf7493ae admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:51.254220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 840/1643] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:51.259040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28063e45-a32b-402d-8510-21c5b7901e52 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:51.274652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28063e45-a32b-402d-8510-21c5b7901e52 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:51.282244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df468d1c-cc26-483a-8ed9-3bffb8302aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:51.282929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df468d1c-cc26-483a-8ed9-3bffb8302aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.283613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df468d1c-cc26-483a-8ed9-3bffb8302aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.284064 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df468d1c-cc26-483a-8ed9-3bffb8302aeb admin admin] PATH_INFO: `/v3/projects/e0a5143630834274bd15411a2d6f9374/users/63854fc4b5124ea39c8da644b0700a33/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.288363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:51.311626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df468d1c-cc26-483a-8ed9-3bffb8302aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a5143630834274bd15411a2d6f9374, user_id=63854fc4b5124ea39c8da644b0700a33, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:51.315276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df468d1c-cc26-483a-8ed9-3bffb8302aeb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:51.345667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 804/1644] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:51 2022] PUT /identity/v3/projects/e0a5143630834274bd15411a2d6f9374/users/63854fc4b5124ea39c8da644b0700a33/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:51.361968 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab39b9-7b49-4356-a16d-b969b3b69e62 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.362264 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab39b9-7b49-4356-a16d-b969b3b69e62 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.365682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab39b9-7b49-4356-a16d-b969b3b69e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.423549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c2ab39b9-7b49-4356-a16d-b969b3b69e62 None None] MFA Rules not processed for user `63854fc4b5124ea39c8da644b0700a33`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:51.427410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c2ab39b9-7b49-4356-a16d-b969b3b69e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:51.440295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:51.511219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df468d1c-cc26-483a-8ed9-3bffb8302aeb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:51.513213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-df468d1c-cc26-483a-8ed9-3bffb8302aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:51.582564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 841/1645] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:51.588042 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab39b9-7b49-4356-a16d-b969b3b69e62 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:51.627160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df468d1c-cc26-483a-8ed9-3bffb8302aeb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:51.633250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78682939-6359-4ef0-a7c8-029aae32a638 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:51.633951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682939-6359-4ef0-a7c8-029aae32a638 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.634230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682939-6359-4ef0-a7c8-029aae32a638 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.634503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682939-6359-4ef0-a7c8-029aae32a638 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.644674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78682939-6359-4ef0-a7c8-029aae32a638 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:51.646283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78682939-6359-4ef0-a7c8-029aae32a638 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:51.657816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 805/1646] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:51.669955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ab5dc-cecc-414f-9d81-e41cd700aa39 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.669955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ab5dc-cecc-414f-9d81-e41cd700aa39 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.670434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ab5dc-cecc-414f-9d81-e41cd700aa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.683464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab39b9-7b49-4356-a16d-b969b3b69e62 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:51.690259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30cbe4-1c40-463d-bb11-64483cb6a235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:51.691155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30cbe4-1c40-463d-bb11-64483cb6a235 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.691428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30cbe4-1c40-463d-bb11-64483cb6a235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.691691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30cbe4-1c40-463d-bb11-64483cb6a235 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.692383 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30cbe4-1c40-463d-bb11-64483cb6a235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:51.694284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30cbe4-1c40-463d-bb11-64483cb6a235 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:51.708233 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-035ab5dc-cecc-414f-9d81-e41cd700aa39 None None] MFA Rules not processed for user `7706eb2365ab4a7f8b28711feae3c22f`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:51.709456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-035ab5dc-cecc-414f-9d81-e41cd700aa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:51.732491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 842/1647] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:51 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:51.733307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30cbe4-1c40-463d-bb11-64483cb6a235 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:51.809717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 806/1648] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:51.814857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30cbe4-1c40-463d-bb11-64483cb6a235 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:51.815153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ab5dc-cecc-414f-9d81-e41cd700aa39 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:51.817434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-035ab5dc-cecc-414f-9d81-e41cd700aa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:51.824561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c981f15-d0f7-4bef-bbfe-e60a8dbe018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:51.825748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c981f15-d0f7-4bef-bbfe-e60a8dbe018a admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.826178 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c981f15-d0f7-4bef-bbfe-e60a8dbe018a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.827242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c981f15-d0f7-4bef-bbfe-e60a8dbe018a admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.828188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c981f15-d0f7-4bef-bbfe-e60a8dbe018a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:51.830986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c981f15-d0f7-4bef-bbfe-e60a8dbe018a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:51.837594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ab5dc-cecc-414f-9d81-e41cd700aa39 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:51.845341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b23ec9-0875-4354-b39f-b30341a6537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:51.847215 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b23ec9-0875-4354-b39f-b30341a6537b admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.848717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b23ec9-0875-4354-b39f-b30341a6537b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.849337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b23ec9-0875-4354-b39f-b30341a6537b admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.850476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b23ec9-0875-4354-b39f-b30341a6537b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:51.855402 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b23ec9-0875-4354-b39f-b30341a6537b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:51.873429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 843/1649] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:51 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:51.878482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c981f15-d0f7-4bef-bbfe-e60a8dbe018a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:51.894837 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c981f15-d0f7-4bef-bbfe-e60a8dbe018a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:51.901997 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b123cb-9cd3-49dc-88bb-440d4983b10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:51.902871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b123cb-9cd3-49dc-88bb-440d4983b10e admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:51.903450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b123cb-9cd3-49dc-88bb-440d4983b10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:51.903729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b123cb-9cd3-49dc-88bb-440d4983b10e admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:51.904944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b123cb-9cd3-49dc-88bb-440d4983b10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:51.906287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 807/1650] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:51 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:51.906791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b123cb-9cd3-49dc-88bb-440d4983b10e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:51.907345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b23ec9-0875-4354-b39f-b30341a6537b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:51.929745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 844/1651] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:51.931384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b123cb-9cd3-49dc-88bb-440d4983b10e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:51.946378 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b123cb-9cd3-49dc-88bb-440d4983b10e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:51.952517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:51.968722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:52.004977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b23ec9-0875-4354-b39f-b30341a6537b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:52.012635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9df4c-3ff1-419c-a0ed-bfbbb001b1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:52.013646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9df4c-3ff1-419c-a0ed-bfbbb001b1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:52.014260 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9df4c-3ff1-419c-a0ed-bfbbb001b1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:52.014668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9df4c-3ff1-419c-a0ed-bfbbb001b1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:52.015744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9df4c-3ff1-419c-a0ed-bfbbb001b1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:52.018463 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9df4c-3ff1-419c-a0ed-bfbbb001b1f6 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:52.032193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa43221-119d-44b0-8d65-d5f59ed65176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:52.032863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa43221-119d-44b0-8d65-d5f59ed65176 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:52.033506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa43221-119d-44b0-8d65-d5f59ed65176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:52.033726 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa43221-119d-44b0-8d65-d5f59ed65176 admin admin] PATH_INFO: `/v3/projects/bbdfae7146bb4256b3920e7fd324d9da/users/36d53ebd47374b5281e43c8913918c06/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:52.059059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa43221-119d-44b0-8d65-d5f59ed65176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbdfae7146bb4256b3920e7fd324d9da, user_id=36d53ebd47374b5281e43c8913918c06, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:52.062787 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa43221-119d-44b0-8d65-d5f59ed65176 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:52.064207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 808/1652] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:51 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:52.068356 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9df4c-3ff1-419c-a0ed-bfbbb001b1f6 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:52.082463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 845/1653] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:51 2022] PUT /identity/v3/projects/bbdfae7146bb4256b3920e7fd324d9da/users/36d53ebd47374b5281e43c8913918c06/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:52.089480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9df4c-3ff1-419c-a0ed-bfbbb001b1f6 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:52.099937 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa43221-119d-44b0-8d65-d5f59ed65176 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:52.177508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158ff9c-6536-494e-9bbb-f2dfb8101497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:52.178303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158ff9c-6536-494e-9bbb-f2dfb8101497 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:52.178690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158ff9c-6536-494e-9bbb-f2dfb8101497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:52.179099 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158ff9c-6536-494e-9bbb-f2dfb8101497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:52.194348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158ff9c-6536-494e-9bbb-f2dfb8101497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:52.194348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158ff9c-6536-494e-9bbb-f2dfb8101497 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:52.195043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 809/1654] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:55:52.198902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158ff9c-6536-494e-9bbb-f2dfb8101497 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:52.210811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158ff9c-6536-494e-9bbb-f2dfb8101497 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:52.211815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa43221-119d-44b0-8d65-d5f59ed65176 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:52.217181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d0d4c-8c30-41fa-9ebf-44ac50af7f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:52.218414 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d0d4c-8c30-41fa-9ebf-44ac50af7f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:52.218875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d0d4c-8c30-41fa-9ebf-44ac50af7f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:52.219058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fec888-cfe7-49d0-b86c-b355ce623212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:52.219348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d0d4c-8c30-41fa-9ebf-44ac50af7f5b admin admin] PATH_INFO: `/v3/projects/e6d45369785e4cd9b6acc04670162be5/users/7c2c0cc62ce44fbfae34d9b149ef134e/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:52.220035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec888-cfe7-49d0-b86c-b355ce623212 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:52.220449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec888-cfe7-49d0-b86c-b355ce623212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:52.220911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec888-cfe7-49d0-b86c-b355ce623212 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:52.221744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fec888-cfe7-49d0-b86c-b355ce623212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:52.223561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fec888-cfe7-49d0-b86c-b355ce623212 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:52.241780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d0d4c-8c30-41fa-9ebf-44ac50af7f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d45369785e4cd9b6acc04670162be5, user_id=7c2c0cc62ce44fbfae34d9b149ef134e, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:52.242895 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 846/1655] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:52.244211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fec888-cfe7-49d0-b86c-b355ce623212 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:52.244329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d0d4c-8c30-41fa-9ebf-44ac50af7f5b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:52.259381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fec888-cfe7-49d0-b86c-b355ce623212 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:52.272868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 810/1656] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:52 2022] PUT /identity/v3/projects/e6d45369785e4cd9b6acc04670162be5/users/7c2c0cc62ce44fbfae34d9b149ef134e/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:52.278034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bd0cd-1149-4aad-9259-61c47aa83386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:52.279333 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bd0cd-1149-4aad-9259-61c47aa83386 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:52.279624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bd0cd-1149-4aad-9259-61c47aa83386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:52.280500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d0d4c-8c30-41fa-9ebf-44ac50af7f5b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:52.281624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bd0cd-1149-4aad-9259-61c47aa83386 admin admin] PATH_INFO: `/v3/projects/bbdfae7146bb4256b3920e7fd324d9da/users/36d53ebd47374b5281e43c8913918c06/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:52.288337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bd0cd-1149-4aad-9259-61c47aa83386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbdfae7146bb4256b3920e7fd324d9da, user_id=36d53ebd47374b5281e43c8913918c06, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:52.297300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bd0cd-1149-4aad-9259-61c47aa83386 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:52.329095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 847/1657] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:52 2022] PUT /identity/v3/projects/bbdfae7146bb4256b3920e7fd324d9da/users/36d53ebd47374b5281e43c8913918c06/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:52.340460 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd0baf-d0cf-4736-8978-15a5f713424d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:52.340743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd0baf-d0cf-4736-8978-15a5f713424d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:52.341025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd0baf-d0cf-4736-8978-15a5f713424d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:52.408277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4fbd0baf-d0cf-4736-8978-15a5f713424d None None] MFA Rules not processed for user `36d53ebd47374b5281e43c8913918c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:52.410043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4fbd0baf-d0cf-4736-8978-15a5f713424d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:52.464421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d0d4c-8c30-41fa-9ebf-44ac50af7f5b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:52.542203 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:52.574758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc75b7-a9fa-4e50-b6ed-924fcdfa4ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:52.575455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc75b7-a9fa-4e50-b6ed-924fcdfa4ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:52.575820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc75b7-a9fa-4e50-b6ed-924fcdfa4ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:52.576246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc75b7-a9fa-4e50-b6ed-924fcdfa4ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:52.577166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc75b7-a9fa-4e50-b6ed-924fcdfa4ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:52.578897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc75b7-a9fa-4e50-b6ed-924fcdfa4ad2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:52.595962 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 848/1658] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:52.596206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 811/1659] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:52.596299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:52.601114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd0baf-d0cf-4736-8978-15a5f713424d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:52.616305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd0baf-d0cf-4736-8978-15a5f713424d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:52.622916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d8363-9fb4-4613-9785-aa9d220eb3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:52.623932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d8363-9fb4-4613-9785-aa9d220eb3fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:52.624269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d8363-9fb4-4613-9785-aa9d220eb3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:52.624584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d8363-9fb4-4613-9785-aa9d220eb3fe admin admin] PATH_INFO: `/v3/projects/e6d45369785e4cd9b6acc04670162be5/users/7c2c0cc62ce44fbfae34d9b149ef134e/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:52.627091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d8363-9fb4-4613-9785-aa9d220eb3fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d45369785e4cd9b6acc04670162be5, user_id=7c2c0cc62ce44fbfae34d9b149ef134e, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:52.630127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d8363-9fb4-4613-9785-aa9d220eb3fe admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:52.640784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc75b7-a9fa-4e50-b6ed-924fcdfa4ad2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:52.646753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 849/1660] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:52 2022] PUT /identity/v3/projects/e6d45369785e4cd9b6acc04670162be5/users/7c2c0cc62ce44fbfae34d9b149ef134e/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:52.725989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc75b7-a9fa-4e50-b6ed-924fcdfa4ad2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:52.809928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2c597-da5a-43d7-b081-cd454192f760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:52.810901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2c597-da5a-43d7-b081-cd454192f760 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:52.811275 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2c597-da5a-43d7-b081-cd454192f760 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:52.811638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2c597-da5a-43d7-b081-cd454192f760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:52.905030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2c597-da5a-43d7-b081-cd454192f760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:52.906755 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2c597-da5a-43d7-b081-cd454192f760 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:52.926404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 812/1661] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:53.097609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d8363-9fb4-4613-9785-aa9d220eb3fe admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:53.099288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ac3d8363-9fb4-4613-9785-aa9d220eb3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:53.112881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d8363-9fb4-4613-9785-aa9d220eb3fe admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:53.117871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5368ec2-405b-4bce-8c22-33693f8bbcb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:53.118791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5368ec2-405b-4bce-8c22-33693f8bbcb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:53.119059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5368ec2-405b-4bce-8c22-33693f8bbcb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:53.119329 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5368ec2-405b-4bce-8c22-33693f8bbcb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:53.202810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5368ec2-405b-4bce-8c22-33693f8bbcb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:53.204363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5368ec2-405b-4bce-8c22-33693f8bbcb1 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:53.216971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 850/1662] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:53.570059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2c597-da5a-43d7-b081-cd454192f760 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:53.621791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:53.637881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2c597-da5a-43d7-b081-cd454192f760 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:53.642476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:53.643138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:53.643431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:53.643824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] PATH_INFO: `/v3/users/63854fc4b5124ea39c8da644b0700a33` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:53.645024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63854fc4b5124ea39c8da644b0700a33)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:53.646931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:53.697922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63854fc4b5124ea39c8da644b0700a33'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:53.698593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63854fc4b5124ea39c8da644b0700a33'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:53.709723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63854fc4b5124ea39c8da644b0700a33'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:53.710489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63854fc4b5124ea39c8da644b0700a33'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:53.727602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e2203d57-7b12-4656-9c2b-1593573ae178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63854fc4b5124ea39c8da644b0700a33'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:53.743961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 813/1663] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:53 2022] DELETE /identity/v3/users/63854fc4b5124ea39c8da644b0700a33 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:53.749342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13632013-4b57-46d1-bac6-326929fbe524 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:53.749565 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13632013-4b57-46d1-bac6-326929fbe524 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:53.749932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13632013-4b57-46d1-bac6-326929fbe524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:53.769068 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d65a-292d-4c92-8935-8aa4d1a2fa55 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:53.769353 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d65a-292d-4c92-8935-8aa4d1a2fa55 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:53.769628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d65a-292d-4c92-8935-8aa4d1a2fa55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:53.774576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-13632013-4b57-46d1-bac6-326929fbe524 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:53.776086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-13632013-4b57-46d1-bac6-326929fbe524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:53.801386 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c6f0d65a-292d-4c92-8935-8aa4d1a2fa55 None None] MFA Rules not processed for user `7c2c0cc62ce44fbfae34d9b149ef134e`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:53.802605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c6f0d65a-292d-4c92-8935-8aa4d1a2fa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:53.865938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 851/1664] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:53.878897 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13632013-4b57-46d1-bac6-326929fbe524 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:53.881961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-13632013-4b57-46d1-bac6-326929fbe524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:53.909921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 814/1665] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:53.923700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0d65a-292d-4c92-8935-8aa4d1a2fa55 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:53.925228 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c6f0d65a-292d-4c92-8935-8aa4d1a2fa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:53.974740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13632013-4b57-46d1-bac6-326929fbe524 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:53.980116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a2ca9-b054-4784-9fb2-19f08db54b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:53.980737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a2ca9-b054-4784-9fb2-19f08db54b5a service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:53.981001 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a2ca9-b054-4784-9fb2-19f08db54b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:53.981343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a2ca9-b054-4784-9fb2-19f08db54b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:53.991485 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0d65a-292d-4c92-8935-8aa4d1a2fa55 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:54.002261 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d340cfd-eecd-4d12-8156-65459644afdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:54.003249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a2ca9-b054-4784-9fb2-19f08db54b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:54.004833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a2ca9-b054-4784-9fb2-19f08db54b5a service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:54.005544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340cfd-eecd-4d12-8156-65459644afdc service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:54.005917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340cfd-eecd-4d12-8156-65459644afdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:54.006306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340cfd-eecd-4d12-8156-65459644afdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:54.010024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2d340cfd-eecd-4d12-8156-65459644afdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:54.017137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:54.022177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 852/1666] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:55:54.106700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a2ca9-b054-4784-9fb2-19f08db54b5a service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:54.108868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5f7a2ca9-b054-4784-9fb2-19f08db54b5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:54.120442 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d340cfd-eecd-4d12-8156-65459644afdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:54.123105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d340cfd-eecd-4d12-8156-65459644afdc service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:54.141807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a2ca9-b054-4784-9fb2-19f08db54b5a service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:54.147918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 815/1667] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:55:54.163028 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:54.164630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:54.166871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:54.167336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] PATH_INFO: `/v3/users/89097ec7688b47e19c16a6b3dc4b2765` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:54.172549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89097ec7688b47e19c16a6b3dc4b2765)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:54.183021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:54.243617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:54.305682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89097ec7688b47e19c16a6b3dc4b2765'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.306466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89097ec7688b47e19c16a6b3dc4b2765'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.328661 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89097ec7688b47e19c16a6b3dc4b2765'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.331124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89097ec7688b47e19c16a6b3dc4b2765'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.359979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89097ec7688b47e19c16a6b3dc4b2765'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.403933 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 853/1668] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:54 2022] DELETE /identity/v3/users/89097ec7688b47e19c16a6b3dc4b2765 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:54.409668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a11b81-e12e-4418-82bf-b6d9ba91417b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:54.409872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a11b81-e12e-4418-82bf-b6d9ba91417b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:54.410155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a11b81-e12e-4418-82bf-b6d9ba91417b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:54.445058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:54.447305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:54.465151 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c3a11b81-e12e-4418-82bf-b6d9ba91417b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:54.469357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c3a11b81-e12e-4418-82bf-b6d9ba91417b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:54.578072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 816/1669] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:54.591090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a11b81-e12e-4418-82bf-b6d9ba91417b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:54.594104 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c3a11b81-e12e-4418-82bf-b6d9ba91417b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:54.622243 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60744c12-60ce-4ebc-aa17-12e562a3fbe9 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:54.635059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:54.637717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:54.638000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:54.638273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] PATH_INFO: `/v3/users/291bf620737e4ddfad7289876f7a6c50` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:54.641380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] RBAC: Authorizing `identity:delete_user(user_id=291bf620737e4ddfad7289876f7a6c50)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:54.643917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:54.672735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a11b81-e12e-4418-82bf-b6d9ba91417b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:54.681764 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068158f-8ee8-4710-a62d-86bd3395e52f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:54.685313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068158f-8ee8-4710-a62d-86bd3395e52f service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:54.685813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068158f-8ee8-4710-a62d-86bd3395e52f service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:54.686237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068158f-8ee8-4710-a62d-86bd3395e52f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:54.725388 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068158f-8ee8-4710-a62d-86bd3395e52f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:54.727230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068158f-8ee8-4710-a62d-86bd3395e52f service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:54.741636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 817/1670] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:54.742921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068158f-8ee8-4710-a62d-86bd3395e52f service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:54.744394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b068158f-8ee8-4710-a62d-86bd3395e52f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:54.758494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068158f-8ee8-4710-a62d-86bd3395e52f service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:54.763729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:54.764550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:54.764944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:54.765445 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] PATH_INFO: `/v3/projects/e0a5143630834274bd15411a2d6f9374` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:54.766581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a5143630834274bd15411a2d6f9374)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:54.768613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:54.783978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291bf620737e4ddfad7289876f7a6c50'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.787206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291bf620737e4ddfad7289876f7a6c50'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.806061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291bf620737e4ddfad7289876f7a6c50'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.807001 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291bf620737e4ddfad7289876f7a6c50'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.836786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-e54a1cfd-a8d5-4752-8b88-09492e6f453f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291bf620737e4ddfad7289876f7a6c50'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.838770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:54.846906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a5143630834274bd15411a2d6f9374', 'request_id': 'req-25befb10-ccc1-4298-893f-a49aae7ce71c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.859426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a5143630834274bd15411a2d6f9374', 'request_id': 'req-25befb10-ccc1-4298-893f-a49aae7ce71c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.868644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 854/1671] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:54 2022] DELETE /identity/v3/users/291bf620737e4ddfad7289876f7a6c50 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:55:54.872487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a5143630834274bd15411a2d6f9374', 'request_id': 'req-25befb10-ccc1-4298-893f-a49aae7ce71c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.875339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cbdec-4816-429c-a129-7543c482190c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:54.876429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cbdec-4816-429c-a129-7543c482190c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:54.876656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cbdec-4816-429c-a129-7543c482190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:54.891404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] The token cache is being invalidate because project e0a5143630834274bd15411a2d6f9374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:54.891900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.892485 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:54.894427 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 818/1672] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:54 2022] DELETE /identity/v3/projects/e0a5143630834274bd15411a2d6f9374 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:54.898187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:54.901112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:54.908585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2d9cbdec-4816-429c-a129-7543c482190c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:54.910187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2d9cbdec-4816-429c-a129-7543c482190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:54.926431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:55.010018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 855/1673] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:55.017212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25befb10-ccc1-4298-893f-a49aae7ce71c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:55.023689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:55.024466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:55.024752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:55.025147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] PATH_INFO: `/v3/users/36d53ebd47374b5281e43c8913918c06` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:55.026404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d53ebd47374b5281e43c8913918c06)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:55.029223 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:55.030479 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cbdec-4816-429c-a129-7543c482190c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:55.034395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2d9cbdec-4816-429c-a129-7543c482190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:55.107197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d53ebd47374b5281e43c8913918c06'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:55.107703 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d53ebd47374b5281e43c8913918c06'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:55.118922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d53ebd47374b5281e43c8913918c06'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:55.119648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d53ebd47374b5281e43c8913918c06'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:55.129090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cbdec-4816-429c-a129-7543c482190c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:55.136475 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a6822-6323-4caf-be7b-29d7fbb4496e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:55.137201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a6822-6323-4caf-be7b-29d7fbb4496e service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:55.137728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a6822-6323-4caf-be7b-29d7fbb4496e service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:55.138115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a6822-6323-4caf-be7b-29d7fbb4496e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:55.139234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d53ebd47374b5281e43c8913918c06'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:55.141294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-eb4a6822-6323-4caf-be7b-29d7fbb4496e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:55.182936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 819/1674] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:54 2022] DELETE /identity/v3/users/36d53ebd47374b5281e43c8913918c06 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:55.189358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a6822-6323-4caf-be7b-29d7fbb4496e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:55.193756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a6822-6323-4caf-be7b-29d7fbb4496e service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:55.221123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 856/1675] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:55.611253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:55.625844 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:55.741608 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:55.765135 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df649a-bda9-4e86-95e3-8455a0b61dc0 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:55.769876 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:55.770527 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:55.770800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:55.771067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] PATH_INFO: `/v3/projects/7088c0a98f0f40b8b1704fd601ae8810` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:55.772229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7088c0a98f0f40b8b1704fd601ae8810)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:55.774587 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:55.821273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a6822-6323-4caf-be7b-29d7fbb4496e service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:55.823721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-eb4a6822-6323-4caf-be7b-29d7fbb4496e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:55.841785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7088c0a98f0f40b8b1704fd601ae8810', 'request_id': 'req-e5128f71-175a-49f1-a040-9574aebd1b75'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:55.850018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7088c0a98f0f40b8b1704fd601ae8810', 'request_id': 'req-e5128f71-175a-49f1-a040-9574aebd1b75'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:55.864080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7088c0a98f0f40b8b1704fd601ae8810', 'request_id': 'req-e5128f71-175a-49f1-a040-9574aebd1b75'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:55.887907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] The token cache is being invalidate because project 7088c0a98f0f40b8b1704fd601ae8810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:55.888122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:55.889615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:55.893407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 820/1676] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:55 2022] DELETE /identity/v3/projects/7088c0a98f0f40b8b1704fd601ae8810 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:55.897415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:55.900102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:55.969879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a6822-6323-4caf-be7b-29d7fbb4496e service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:56.042267 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5128f71-175a-49f1-a040-9574aebd1b75 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:56.050519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:56.051177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.051777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.052106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] PATH_INFO: `/v3/users/0ed4a92fe9ef415eba52f4cea63556ce` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.053276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed4a92fe9ef415eba52f4cea63556ce)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:56.055276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:56.059183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d0d19-f43c-476f-807a-27f23bd80301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:56.059814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d0d19-f43c-476f-807a-27f23bd80301 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.060392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d0d19-f43c-476f-807a-27f23bd80301 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.060686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d0d19-f43c-476f-807a-27f23bd80301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.063400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-547d0d19-f43c-476f-807a-27f23bd80301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.102230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:56.107591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed4a92fe9ef415eba52f4cea63556ce'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.108466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed4a92fe9ef415eba52f4cea63556ce'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.116629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed4a92fe9ef415eba52f4cea63556ce'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.117669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed4a92fe9ef415eba52f4cea63556ce'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.129926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed4a92fe9ef415eba52f4cea63556ce'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.134084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d0d19-f43c-476f-807a-27f23bd80301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:56.135647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d0d19-f43c-476f-807a-27f23bd80301 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:56.139468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:56.144316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 821/1677] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:55:55 2022] DELETE /identity/v3/users/0ed4a92fe9ef415eba52f4cea63556ce => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:56.146206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 857/1678] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:55:56.147111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d0d19-f43c-476f-807a-27f23bd80301 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:56.149168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:56.151080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.158502 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-547d0d19-f43c-476f-807a-27f23bd80301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.174875 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d0d19-f43c-476f-807a-27f23bd80301 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:56.249587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:56.250526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.250862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.251745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] PATH_INFO: `/v3/projects/bbdfae7146bb4256b3920e7fd324d9da` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.257827 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d99ad-fe3c-46ad-b9be-e5bf455a24b2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:56.262331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbdfae7146bb4256b3920e7fd324d9da)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:56.267610 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:56.268602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.269725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:56.270181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.270861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] PATH_INFO: `/v3/projects/640684dd19c64642a6984eb8eafa64b3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.274745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=640684dd19c64642a6984eb8eafa64b3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:56.276829 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:56.369905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbdfae7146bb4256b3920e7fd324d9da', 'request_id': 'req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.376457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '640684dd19c64642a6984eb8eafa64b3', 'request_id': 'req-12e0f5b7-7212-4d00-86e6-81823997c596'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.379465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbdfae7146bb4256b3920e7fd324d9da', 'request_id': 'req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.385061 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '640684dd19c64642a6984eb8eafa64b3', 'request_id': 'req-12e0f5b7-7212-4d00-86e6-81823997c596'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.398056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbdfae7146bb4256b3920e7fd324d9da', 'request_id': 'req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.403122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '640684dd19c64642a6984eb8eafa64b3', 'request_id': 'req-12e0f5b7-7212-4d00-86e6-81823997c596'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.416847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] The token cache is being invalidate because project bbdfae7146bb4256b3920e7fd324d9da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:56.416847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.417730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7abc6ac9-28aa-4f06-92f7-7fc86b60e94e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.424269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 858/1679] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:56 2022] DELETE /identity/v3/projects/bbdfae7146bb4256b3920e7fd324d9da => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:56.426894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] The token cache is being invalidate because project 640684dd19c64642a6984eb8eafa64b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:56.427199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.427949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-12e0f5b7-7212-4d00-86e6-81823997c596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.429488 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 822/1680] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:56 2022] DELETE /identity/v3/projects/640684dd19c64642a6984eb8eafa64b3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:56.430464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fd014-e45e-49b2-acb0-3cc7433b478f None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.431164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fd014-e45e-49b2-acb0-3cc7433b478f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.431803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fd014-e45e-49b2-acb0-3cc7433b478f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.436204 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4be71-16d3-4f8f-a707-3ebb332909fd None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.436492 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4be71-16d3-4f8f-a707-3ebb332909fd None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.436771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4be71-16d3-4f8f-a707-3ebb332909fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.454824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-780fd014-e45e-49b2-acb0-3cc7433b478f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:56.456500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-780fd014-e45e-49b2-acb0-3cc7433b478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.470720 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-e2e4be71-16d3-4f8f-a707-3ebb332909fd None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:56.472355 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e2e4be71-16d3-4f8f-a707-3ebb332909fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.532624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:56.547951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 859/1681] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:56.549988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fd014-e45e-49b2-acb0-3cc7433b478f None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:56.551286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-780fd014-e45e-49b2-acb0-3cc7433b478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.564078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 823/1682] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:56.565695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66038882-3918-45d1-802d-f9f90d8a504a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.566197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66038882-3918-45d1-802d-f9f90d8a504a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.566661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66038882-3918-45d1-802d-f9f90d8a504a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.573325 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fd014-e45e-49b2-acb0-3cc7433b478f None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:56.578386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:56.579055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.579332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.579601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] PATH_INFO: `/v3/projects/ba717b1f44c542ed8b261f2b8455c57a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.580960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba717b1f44c542ed8b261f2b8455c57a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:56.583138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:56.590599 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-66038882-3918-45d1-802d-f9f90d8a504a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:56.592082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-66038882-3918-45d1-802d-f9f90d8a504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.602757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 824/1683] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:56.604734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6dd37-3944-41cf-bbb8-872dbeb462a9 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.605169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6dd37-3944-41cf-bbb8-872dbeb462a9 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.605598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6dd37-3944-41cf-bbb8-872dbeb462a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.630033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-e5a6dd37-3944-41cf-bbb8-872dbeb462a9 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:56.631428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e5a6dd37-3944-41cf-bbb8-872dbeb462a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.662174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba717b1f44c542ed8b261f2b8455c57a', 'request_id': 'req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.673033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba717b1f44c542ed8b261f2b8455c57a', 'request_id': 'req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.688891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba717b1f44c542ed8b261f2b8455c57a', 'request_id': 'req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.690564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:56.716557 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] The token cache is being invalidate because project ba717b1f44c542ed8b261f2b8455c57a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:55:56.716921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.718735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:55:56.722107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 860/1684] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:55:56 2022] DELETE /identity/v3/projects/ba717b1f44c542ed8b261f2b8455c57a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:56.725381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:56.733032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.775077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 825/1685] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:56.776880 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def909c8-27a3-4821-a04d-b7a91f3bf6a8 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.777399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def909c8-27a3-4821-a04d-b7a91f3bf6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.777882 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def909c8-27a3-4821-a04d-b7a91f3bf6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.808913 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-def909c8-27a3-4821-a04d-b7a91f3bf6a8 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:56.810742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-def909c8-27a3-4821-a04d-b7a91f3bf6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.823129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 826/1686] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:56.824177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def909c8-27a3-4821-a04d-b7a91f3bf6a8 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:56.844537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def909c8-27a3-4821-a04d-b7a91f3bf6a8 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:56.853388 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08570590-7e36-4ae0-8aa4-0813d2312c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:56.854237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08570590-7e36-4ae0-8aa4-0813d2312c0a admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.855433 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08570590-7e36-4ae0-8aa4-0813d2312c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.856362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08570590-7e36-4ae0-8aa4-0813d2312c0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.859839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08570590-7e36-4ae0-8aa4-0813d2312c0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:56.860256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08570590-7e36-4ae0-8aa4-0813d2312c0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:56.862701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08570590-7e36-4ae0-8aa4-0813d2312c0a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:56.864547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f6bc6-cc1c-4891-be6b-e287a0a28bd7 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:56.872667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296524b-123a-4439-9b99-831c7df235d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:56.873301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296524b-123a-4439-9b99-831c7df235d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.873581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296524b-123a-4439-9b99-831c7df235d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.873914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296524b-123a-4439-9b99-831c7df235d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.874550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296524b-123a-4439-9b99-831c7df235d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:56.874875 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296524b-123a-4439-9b99-831c7df235d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:56.877067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296524b-123a-4439-9b99-831c7df235d6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:56.895549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 861/1687] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:56.895549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7c963-80b6-4b7a-9a75-a09b5a4af873 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.895549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7c963-80b6-4b7a-9a75-a09b5a4af873 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.895549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7c963-80b6-4b7a-9a75-a09b5a4af873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.895944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 827/1688] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:56.897341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08570590-7e36-4ae0-8aa4-0813d2312c0a admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:56.918543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08570590-7e36-4ae0-8aa4-0813d2312c0a admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:56.925348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f0e7c963-80b6-4b7a-9a75-a09b5a4af873 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:56.929529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f0e7c963-80b6-4b7a-9a75-a09b5a4af873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:56.931804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ce373-6d41-4664-a4d4-c022587803de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:56.932489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ce373-6d41-4664-a4d4-c022587803de admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.932776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ce373-6d41-4664-a4d4-c022587803de admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.933076 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ce373-6d41-4664-a4d4-c022587803de admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.933723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ce373-6d41-4664-a4d4-c022587803de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:56.935773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ce373-6d41-4664-a4d4-c022587803de admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:56.942775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 862/1689] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:56.942775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7c963-80b6-4b7a-9a75-a09b5a4af873 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:56.953393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7c963-80b6-4b7a-9a75-a09b5a4af873 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:56.962281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273fca5-a49b-4f2c-9eab-4f4a9f6ee217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:56.963226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273fca5-a49b-4f2c-9eab-4f4a9f6ee217 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:56.963509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273fca5-a49b-4f2c-9eab-4f4a9f6ee217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:56.963864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273fca5-a49b-4f2c-9eab-4f4a9f6ee217 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:56.964851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273fca5-a49b-4f2c-9eab-4f4a9f6ee217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:56.968289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273fca5-a49b-4f2c-9eab-4f4a9f6ee217 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:56.997537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 828/1690] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:56 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:55:56.998416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ce373-6d41-4664-a4d4-c022587803de admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.033889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 863/1691] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:56 2022] POST /identity/v3/projects => generated 369 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:57.033889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273fca5-a49b-4f2c-9eab-4f4a9f6ee217 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.122744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ce373-6d41-4664-a4d4-c022587803de admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.133003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:57.152026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273fca5-a49b-4f2c-9eab-4f4a9f6ee217 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.161151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2cf31-703a-49dd-a106-b916a8df141a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.164295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2cf31-703a-49dd-a106-b916a8df141a admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.164935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2cf31-703a-49dd-a106-b916a8df141a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.165260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2cf31-703a-49dd-a106-b916a8df141a admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.166389 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2cf31-703a-49dd-a106-b916a8df141a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.168597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2cf31-703a-49dd-a106-b916a8df141a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.209887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 864/1692] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:57 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:57.209887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2cf31-703a-49dd-a106-b916a8df141a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.218040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18f948-2499-4477-9b13-cfb1cbf7858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.219237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18f948-2499-4477-9b13-cfb1cbf7858e admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.219522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18f948-2499-4477-9b13-cfb1cbf7858e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.219792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18f948-2499-4477-9b13-cfb1cbf7858e admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.220489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18f948-2499-4477-9b13-cfb1cbf7858e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:55:57.220789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18f948-2499-4477-9b13-cfb1cbf7858e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.222694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18f948-2499-4477-9b13-cfb1cbf7858e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.231188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2cf31-703a-49dd-a106-b916a8df141a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.240043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 829/1693] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:55:57.240043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18f948-2499-4477-9b13-cfb1cbf7858e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.243656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c628fb-58ec-46ae-93d2-7a05ed6a85bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.244573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c628fb-58ec-46ae-93d2-7a05ed6a85bd admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.244959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c628fb-58ec-46ae-93d2-7a05ed6a85bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.245330 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c628fb-58ec-46ae-93d2-7a05ed6a85bd admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.246095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c628fb-58ec-46ae-93d2-7a05ed6a85bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.248216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c628fb-58ec-46ae-93d2-7a05ed6a85bd admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.254925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18f948-2499-4477-9b13-cfb1cbf7858e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.261962 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8748b1-1583-4fe0-9948-3ec5619d104e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.262961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8748b1-1583-4fe0-9948-3ec5619d104e admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.262961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8748b1-1583-4fe0-9948-3ec5619d104e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.263169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8748b1-1583-4fe0-9948-3ec5619d104e admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.263959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8748b1-1583-4fe0-9948-3ec5619d104e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.265838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8748b1-1583-4fe0-9948-3ec5619d104e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.275635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 865/1694] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:57 2022] POST /identity/v3/users => generated 285 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:57.277408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c628fb-58ec-46ae-93d2-7a05ed6a85bd admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.289032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c628fb-58ec-46ae-93d2-7a05ed6a85bd admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.293061 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 830/1695] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:57.293061 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8748b1-1583-4fe0-9948-3ec5619d104e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.299662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1753e2-fe59-4242-8d28-d4d12dad0456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.300333 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1753e2-fe59-4242-8d28-d4d12dad0456 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.300615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1753e2-fe59-4242-8d28-d4d12dad0456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.301416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1753e2-fe59-4242-8d28-d4d12dad0456 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.301536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1753e2-fe59-4242-8d28-d4d12dad0456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.303441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1753e2-fe59-4242-8d28-d4d12dad0456 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.307784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8748b1-1583-4fe0-9948-3ec5619d104e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.316040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8c932-21a3-40fc-a131-433cbe33ffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.317053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8c932-21a3-40fc-a131-433cbe33ffc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.317339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8c932-21a3-40fc-a131-433cbe33ffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.317661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8c932-21a3-40fc-a131-433cbe33ffc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.318306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8c932-21a3-40fc-a131-433cbe33ffc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.324358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8c932-21a3-40fc-a131-433cbe33ffc2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.345043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 831/1696] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:57.345959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8c932-21a3-40fc-a131-433cbe33ffc2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.358269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 866/1697] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:57 2022] POST /identity/v3/projects => generated 367 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:57.358269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1753e2-fe59-4242-8d28-d4d12dad0456 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.372518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:57.380914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8c932-21a3-40fc-a131-433cbe33ffc2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.475549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1753e2-fe59-4242-8d28-d4d12dad0456 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.475762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d3c29-6af5-4763-8fd1-70a1bb6fac81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.476235 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d3c29-6af5-4763-8fd1-70a1bb6fac81 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.476511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d3c29-6af5-4763-8fd1-70a1bb6fac81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.476849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d3c29-6af5-4763-8fd1-70a1bb6fac81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.477478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d3c29-6af5-4763-8fd1-70a1bb6fac81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.479209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d3c29-6af5-4763-8fd1-70a1bb6fac81 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.484798 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477e14d-a5f0-451d-895f-100f400f7c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.486113 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477e14d-a5f0-451d-895f-100f400f7c3c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.486575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477e14d-a5f0-451d-895f-100f400f7c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.486992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477e14d-a5f0-451d-895f-100f400f7c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.488069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477e14d-a5f0-451d-895f-100f400f7c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.489730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477e14d-a5f0-451d-895f-100f400f7c3c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.495035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 832/1698] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:57.496199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d3c29-6af5-4763-8fd1-70a1bb6fac81 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.511697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 867/1699] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:57.511697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477e14d-a5f0-451d-895f-100f400f7c3c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.515835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d3c29-6af5-4763-8fd1-70a1bb6fac81 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.524459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477e14d-a5f0-451d-895f-100f400f7c3c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.525051 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a1462-0d0c-475d-b4c9-ca00db1220ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.525713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a1462-0d0c-475d-b4c9-ca00db1220ff admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.525984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a1462-0d0c-475d-b4c9-ca00db1220ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.526247 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a1462-0d0c-475d-b4c9-ca00db1220ff admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.526875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a1462-0d0c-475d-b4c9-ca00db1220ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.528891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a1462-0d0c-475d-b4c9-ca00db1220ff admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.541077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b90f44-2af8-4250-8ee8-8ef3b2948c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.541914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b90f44-2af8-4250-8ee8-8ef3b2948c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.542379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b90f44-2af8-4250-8ee8-8ef3b2948c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.542755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b90f44-2af8-4250-8ee8-8ef3b2948c98 admin admin] PATH_INFO: `/v3/projects/3960c95fdbb7477082ec53d4585b4bf6/users/bbb3a5004ee64ad5a074d5e24f05414b/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.564816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b90f44-2af8-4250-8ee8-8ef3b2948c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3960c95fdbb7477082ec53d4585b4bf6, user_id=bbb3a5004ee64ad5a074d5e24f05414b, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.567947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b90f44-2af8-4250-8ee8-8ef3b2948c98 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.578996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 833/1700] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:57 2022] POST /identity/v3/users => generated 284 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:57.584588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a1462-0d0c-475d-b4c9-ca00db1220ff admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.586860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 868/1701] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:57 2022] PUT /identity/v3/projects/3960c95fdbb7477082ec53d4585b4bf6/users/bbb3a5004ee64ad5a074d5e24f05414b/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:57.588026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b90f44-2af8-4250-8ee8-8ef3b2948c98 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.617879 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a1462-0d0c-475d-b4c9-ca00db1220ff admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.714109 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547eba8-fadc-47d1-9112-4978d6d1f13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.715266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547eba8-fadc-47d1-9112-4978d6d1f13c admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.716283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547eba8-fadc-47d1-9112-4978d6d1f13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.716670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547eba8-fadc-47d1-9112-4978d6d1f13c admin admin] PATH_INFO: `/v3/projects/fd58299057494f4d9bf89b7068943f8b/users/342c9e930d834316bcd2997476370d2e/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.736720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b90f44-2af8-4250-8ee8-8ef3b2948c98 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.748835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178afda-3d19-4794-8f8e-8417ac1e624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.749463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178afda-3d19-4794-8f8e-8417ac1e624b admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.749766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178afda-3d19-4794-8f8e-8417ac1e624b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.750039 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178afda-3d19-4794-8f8e-8417ac1e624b admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.750766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178afda-3d19-4794-8f8e-8417ac1e624b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.752591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547eba8-fadc-47d1-9112-4978d6d1f13c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd58299057494f4d9bf89b7068943f8b, user_id=342c9e930d834316bcd2997476370d2e, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.752712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178afda-3d19-4794-8f8e-8417ac1e624b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.756783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547eba8-fadc-47d1-9112-4978d6d1f13c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.767921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 869/1702] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:57.772434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582b7c8-ad6f-4dfb-8f13-85a1e22bb2d8 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.772753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582b7c8-ad6f-4dfb-8f13-85a1e22bb2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.773269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582b7c8-ad6f-4dfb-8f13-85a1e22bb2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.787931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 834/1703] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:57 2022] PUT /identity/v3/projects/fd58299057494f4d9bf89b7068943f8b/users/342c9e930d834316bcd2997476370d2e/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:57.787931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547eba8-fadc-47d1-9112-4978d6d1f13c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.789630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:57.813418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1582b7c8-ad6f-4dfb-8f13-85a1e22bb2d8 None None] MFA Rules not processed for user `bbb3a5004ee64ad5a074d5e24f05414b`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:57.814790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1582b7c8-ad6f-4dfb-8f13-85a1e22bb2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:57.871731 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547eba8-fadc-47d1-9112-4978d6d1f13c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.878871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063d9ba-5f21-49d3-953e-e2bda6998ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.879493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d9ba-5f21-49d3-953e-e2bda6998ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.879772 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d9ba-5f21-49d3-953e-e2bda6998ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.880071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d9ba-5f21-49d3-953e-e2bda6998ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.880725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063d9ba-5f21-49d3-953e-e2bda6998ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.882548 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063d9ba-5f21-49d3-953e-e2bda6998ee9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.892716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:57.898375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 835/1704] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:57.901035 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef4456-8218-4185-847a-a6f423db4cca None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.902368 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef4456-8218-4185-847a-a6f423db4cca None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.903040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef4456-8218-4185-847a-a6f423db4cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.903593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 870/1705] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:57.904536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582b7c8-ad6f-4dfb-8f13-85a1e22bb2d8 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.917446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582b7c8-ad6f-4dfb-8f13-85a1e22bb2d8 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:57.922211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c03c3-b980-41c9-8b26-94da0720ac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:57.922998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c03c3-b980-41c9-8b26-94da0720ac02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:57.923367 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c03c3-b980-41c9-8b26-94da0720ac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:57.923704 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c03c3-b980-41c9-8b26-94da0720ac02 admin admin] PATH_INFO: `/v3/projects/288e179daad74a288313ef24d09b734b/users/802e973e89d84141acb391ede3948767/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:57.937831 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-59ef4456-8218-4185-847a-a6f423db4cca None None] MFA Rules not processed for user `342c9e930d834316bcd2997476370d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:57.939249 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-59ef4456-8218-4185-847a-a6f423db4cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:57.939864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c03c3-b980-41c9-8b26-94da0720ac02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=288e179daad74a288313ef24d09b734b, user_id=802e973e89d84141acb391ede3948767, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:57.943043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c03c3-b980-41c9-8b26-94da0720ac02 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:57.957225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 871/1706] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:57 2022] PUT /identity/v3/projects/288e179daad74a288313ef24d09b734b/users/802e973e89d84141acb391ede3948767/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:57.960834 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c03c3-b980-41c9-8b26-94da0720ac02 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:57.963574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-299c03c3-b980-41c9-8b26-94da0720ac02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:58.030873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 836/1707] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:58.030873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556fac7-67d1-4e9e-83a9-91db3c4ab559 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:58.030873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556fac7-67d1-4e9e-83a9-91db3c4ab559 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:58.030873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556fac7-67d1-4e9e-83a9-91db3c4ab559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:58.048878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c03c3-b980-41c9-8b26-94da0720ac02 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:58.056723 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eeda9c-69b2-4b84-a9e0-e553ac7fc30d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:58.057575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeda9c-69b2-4b84-a9e0-e553ac7fc30d service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:58.058025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeda9c-69b2-4b84-a9e0-e553ac7fc30d service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:58.058713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeda9c-69b2-4b84-a9e0-e553ac7fc30d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:58.082287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a556fac7-67d1-4e9e-83a9-91db3c4ab559 None None] MFA Rules not processed for user `802e973e89d84141acb391ede3948767`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:58.083864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a556fac7-67d1-4e9e-83a9-91db3c4ab559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:58.141948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eeda9c-69b2-4b84-a9e0-e553ac7fc30d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:58.144977 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eeda9c-69b2-4b84-a9e0-e553ac7fc30d service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:58.161473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 872/1708] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:58.164760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eeda9c-69b2-4b84-a9e0-e553ac7fc30d service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:58.167366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-84eeda9c-69b2-4b84-a9e0-e553ac7fc30d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:58.183478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 837/1709] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:55:58.189108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556fac7-67d1-4e9e-83a9-91db3c4ab559 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:58.251097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:58.315595 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556fac7-67d1-4e9e-83a9-91db3c4ab559 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:58.348997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e132-6bbe-4351-9720-7450e519eaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:58.351174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e132-6bbe-4351-9720-7450e519eaf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:58.353248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e132-6bbe-4351-9720-7450e519eaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:58.354094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e132-6bbe-4351-9720-7450e519eaf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:58.355227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82e132-6bbe-4351-9720-7450e519eaf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:58.360368 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82e132-6bbe-4351-9720-7450e519eaf4 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:58.364628 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eeda9c-69b2-4b84-a9e0-e553ac7fc30d service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:58.376768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc9b44-c67c-4d89-9915-bf9bad5aaee7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3566d85c9c4c559ba672269ab38f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:58.377501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc9b44-c67c-4d89-9915-bf9bad5aaee7 service glance] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:58.377661 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc9b44-c67c-4d89-9915-bf9bad5aaee7 service glance] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:58.378012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc9b44-c67c-4d89-9915-bf9bad5aaee7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:58.445028 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 838/1710] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:55:58 2022] POST /identity/v3/projects => generated 365 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:58.451755 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e132-6bbe-4351-9720-7450e519eaf4 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:58.466880 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:58.513948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc9b44-c67c-4d89-9915-bf9bad5aaee7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:58.529271 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc9b44-c67c-4d89-9915-bf9bad5aaee7 service glance] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:58.548184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 873/1711] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:58.559040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e132-6bbe-4351-9720-7450e519eaf4 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:58.582364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eac77-4eb8-4e03-b71c-3bb34deac28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:58.583111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eac77-4eb8-4e03-b71c-3bb34deac28b admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:58.583490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eac77-4eb8-4e03-b71c-3bb34deac28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:58.584004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eac77-4eb8-4e03-b71c-3bb34deac28b admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:58.584665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179eac77-4eb8-4e03-b71c-3bb34deac28b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:58.589396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179eac77-4eb8-4e03-b71c-3bb34deac28b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:58.633474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 839/1712] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:55:58 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:55:58.638458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc9b44-c67c-4d89-9915-bf9bad5aaee7 service glance] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:58.653745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc9b44-c67c-4d89-9915-bf9bad5aaee7 service glance] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:58.659177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e45a89-68d7-4370-8116-fe8f7541d735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:58.660067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e45a89-68d7-4370-8116-fe8f7541d735 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:58.660580 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e45a89-68d7-4370-8116-fe8f7541d735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:58.660828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e45a89-68d7-4370-8116-fe8f7541d735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:58.661607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e45a89-68d7-4370-8116-fe8f7541d735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:58.663941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e45a89-68d7-4370-8116-fe8f7541d735 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:58.681145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eac77-4eb8-4e03-b71c-3bb34deac28b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:58.689299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 874/1713] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:58.693934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eac77-4eb8-4e03-b71c-3bb34deac28b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:58.702513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303fed4-f088-4d7e-98f5-1c9aedf9a5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:58.703360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303fed4-f088-4d7e-98f5-1c9aedf9a5e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:58.703758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303fed4-f088-4d7e-98f5-1c9aedf9a5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:58.704152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303fed4-f088-4d7e-98f5-1c9aedf9a5e1 admin admin] PATH_INFO: `/v3/projects/95e8cc2243fb4b4ea501e7f8db711fc4/users/57322ee29b1c46d293f25ee905220234/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:58.724056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303fed4-f088-4d7e-98f5-1c9aedf9a5e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e8cc2243fb4b4ea501e7f8db711fc4, user_id=57322ee29b1c46d293f25ee905220234, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:58.728956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303fed4-f088-4d7e-98f5-1c9aedf9a5e1 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:58.778857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e45a89-68d7-4370-8116-fe8f7541d735 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:58.786513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 840/1714] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:58 2022] PUT /identity/v3/projects/95e8cc2243fb4b4ea501e7f8db711fc4/users/57322ee29b1c46d293f25ee905220234/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:58.793457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e45a89-68d7-4370-8116-fe8f7541d735 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:58.921052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5744e1b-a06b-4a81-a360-6a5f71cf39f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:58.921865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744e1b-a06b-4a81-a360-6a5f71cf39f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:58.922245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744e1b-a06b-4a81-a360-6a5f71cf39f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:58.923241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744e1b-a06b-4a81-a360-6a5f71cf39f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:58.923624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5744e1b-a06b-4a81-a360-6a5f71cf39f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:58.927404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5744e1b-a06b-4a81-a360-6a5f71cf39f7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:58.944913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 875/1715] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:55:58.949771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303fed4-f088-4d7e-98f5-1c9aedf9a5e1 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:58.960378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303fed4-f088-4d7e-98f5-1c9aedf9a5e1 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:58.968244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea380424-b4b8-4cf8-856c-6a34eea894b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:58.969000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea380424-b4b8-4cf8-856c-6a34eea894b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:58.969338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea380424-b4b8-4cf8-856c-6a34eea894b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:58.969650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea380424-b4b8-4cf8-856c-6a34eea894b3 admin admin] PATH_INFO: `/v3/projects/95e8cc2243fb4b4ea501e7f8db711fc4/users/57322ee29b1c46d293f25ee905220234/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:58.971553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea380424-b4b8-4cf8-856c-6a34eea894b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e8cc2243fb4b4ea501e7f8db711fc4, user_id=57322ee29b1c46d293f25ee905220234, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:58.974434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea380424-b4b8-4cf8-856c-6a34eea894b3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:58.988484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 841/1716] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:55:58 2022] PUT /identity/v3/projects/95e8cc2243fb4b4ea501e7f8db711fc4/users/57322ee29b1c46d293f25ee905220234/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:55:59.000732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb2c77-ef61-41fb-b3f6-7d4072c4e3d7 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:59.001902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb2c77-ef61-41fb-b3f6-7d4072c4e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:59.007266 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb2c77-ef61-41fb-b3f6-7d4072c4e3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:59.046070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-02cb2c77-ef61-41fb-b3f6-7d4072c4e3d7 None None] MFA Rules not processed for user `57322ee29b1c46d293f25ee905220234`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:55:59.047288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-02cb2c77-ef61-41fb-b3f6-7d4072c4e3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:59.101205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:55:59.158736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 876/1717] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:55:59.174163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea380424-b4b8-4cf8-856c-6a34eea894b3 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:59.176251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ea380424-b4b8-4cf8-856c-6a34eea894b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:55:59.256968 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea380424-b4b8-4cf8-856c-6a34eea894b3 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:59.265255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0426c79-fb1d-41e4-a1c0-5c059486d715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:59.265888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0426c79-fb1d-41e4-a1c0-5c059486d715 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:59.266157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0426c79-fb1d-41e4-a1c0-5c059486d715 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:59.266430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0426c79-fb1d-41e4-a1c0-5c059486d715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:59.277598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0426c79-fb1d-41e4-a1c0-5c059486d715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:59.279150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0426c79-fb1d-41e4-a1c0-5c059486d715 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:59.297108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 842/1718] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:55:59.457135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb2c77-ef61-41fb-b3f6-7d4072c4e3d7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:55:59.468435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb2c77-ef61-41fb-b3f6-7d4072c4e3d7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:55:59.472947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03667f3-62d0-42f7-8761-b5fe27fe3027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:55:59.473770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03667f3-62d0-42f7-8761-b5fe27fe3027 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:55:59.474163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03667f3-62d0-42f7-8761-b5fe27fe3027 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:55:59.474582 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03667f3-62d0-42f7-8761-b5fe27fe3027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:55:59.542034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03667f3-62d0-42f7-8761-b5fe27fe3027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:55:59.543458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03667f3-62d0-42f7-8761-b5fe27fe3027 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:55:59.552607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 877/1719] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:55:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:07.334376 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0426c79-fb1d-41e4-a1c0-5c059486d715 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:07.433578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0426c79-fb1d-41e4-a1c0-5c059486d715 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:07.438439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:07.439103 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:07.439448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:07.439624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] PATH_INFO: `/v3/users/802e973e89d84141acb391ede3948767` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:07.440854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=802e973e89d84141acb391ede3948767)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:07.442820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:07.444645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:07.506870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03667f3-62d0-42f7-8761-b5fe27fe3027 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:07.511867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802e973e89d84141acb391ede3948767'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:07.514107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802e973e89d84141acb391ede3948767'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:07.518173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03667f3-62d0-42f7-8761-b5fe27fe3027 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:07.523304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:07.524939 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:07.525275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:07.525622 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] PATH_INFO: `/v3/users/bbb3a5004ee64ad5a074d5e24f05414b` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:07.526294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802e973e89d84141acb391ede3948767'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:07.526779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb3a5004ee64ad5a074d5e24f05414b)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:07.527087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802e973e89d84141acb391ede3948767'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:07.528736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:07.546544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-bde64329-6fcc-44ef-8f05-04108c5d3800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802e973e89d84141acb391ede3948767'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:07.561389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 843/1720] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:07 2022] DELETE /identity/v3/users/802e973e89d84141acb391ede3948767 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:07.565646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ba05f-8dba-48b6-8395-0fe987b652fa None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:07.565931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ba05f-8dba-48b6-8395-0fe987b652fa None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:07.566206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ba05f-8dba-48b6-8395-0fe987b652fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:07.587965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2e1ba05f-8dba-48b6-8395-0fe987b652fa None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:07.589107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2e1ba05f-8dba-48b6-8395-0fe987b652fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:07.596011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb3a5004ee64ad5a074d5e24f05414b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:07.596744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb3a5004ee64ad5a074d5e24f05414b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:07.607610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb3a5004ee64ad5a074d5e24f05414b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:07.608360 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb3a5004ee64ad5a074d5e24f05414b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:07.623048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5b158346-21ae-4524-9db0-21518899c3e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb3a5004ee64ad5a074d5e24f05414b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:07.642145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 878/1721] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:07 2022] DELETE /identity/v3/users/bbb3a5004ee64ad5a074d5e24f05414b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:07.644273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45aa52-3914-47e0-8a76-fe9409a75ab2 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:07.644558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45aa52-3914-47e0-8a76-fe9409a75ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:07.644832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45aa52-3914-47e0-8a76-fe9409a75ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:07.663811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 844/1722] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:07.670737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8d45aa52-3914-47e0-8a76-fe9409a75ab2 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:07.672072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8d45aa52-3914-47e0-8a76-fe9409a75ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:07.680005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ba05f-8dba-48b6-8395-0fe987b652fa None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:07.682925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2e1ba05f-8dba-48b6-8395-0fe987b652fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:07.745294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:07.792411 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ba05f-8dba-48b6-8395-0fe987b652fa None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:07.814721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 879/1723] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:07.827177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45aa52-3914-47e0-8a76-fe9409a75ab2 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:07.829338 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8d45aa52-3914-47e0-8a76-fe9409a75ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:07.874739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:07.925318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd89dd-1c59-4367-a92b-8e22456a06b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:07.926854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd89dd-1c59-4367-a92b-8e22456a06b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:07.927363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd89dd-1c59-4367-a92b-8e22456a06b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:07.927830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd89dd-1c59-4367-a92b-8e22456a06b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:07.931544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45aa52-3914-47e0-8a76-fe9409a75ab2 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:07.953433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07a331-d370-4e6e-ab56-787925cd3c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:07.954071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07a331-d370-4e6e-ab56-787925cd3c22 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:07.954347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07a331-d370-4e6e-ab56-787925cd3c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:07.954611 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07a331-d370-4e6e-ab56-787925cd3c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:07.966511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07a331-d370-4e6e-ab56-787925cd3c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:07.968103 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07a331-d370-4e6e-ab56-787925cd3c22 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:07.971257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd89dd-1c59-4367-a92b-8e22456a06b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:07.973299 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd89dd-1c59-4367-a92b-8e22456a06b8 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:07.980941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 880/1724] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:07.988483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 845/1725] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:08.291678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07a331-d370-4e6e-ab56-787925cd3c22 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:08.293126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ff07a331-d370-4e6e-ab56-787925cd3c22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:08.313114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07a331-d370-4e6e-ab56-787925cd3c22 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:08.318653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:08.319300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:08.319575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:08.319845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] PATH_INFO: `/v3/projects/3960c95fdbb7477082ec53d4585b4bf6` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:08.325187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3960c95fdbb7477082ec53d4585b4bf6)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:08.329183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:08.393732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3960c95fdbb7477082ec53d4585b4bf6', 'request_id': 'req-899adee5-d91e-442f-a3a8-cb7b25498a8e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:08.400971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3960c95fdbb7477082ec53d4585b4bf6', 'request_id': 'req-899adee5-d91e-442f-a3a8-cb7b25498a8e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:08.412715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3960c95fdbb7477082ec53d4585b4bf6', 'request_id': 'req-899adee5-d91e-442f-a3a8-cb7b25498a8e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:08.427487 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] The token cache is being invalidate because project 3960c95fdbb7477082ec53d4585b4bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:08.428302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:08.429762 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-899adee5-d91e-442f-a3a8-cb7b25498a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:08.436561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 881/1726] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:08 2022] DELETE /identity/v3/projects/3960c95fdbb7477082ec53d4585b4bf6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:08.437825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4661-4e8f-472d-b95f-c68f050f0eef None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:08.438237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4661-4e8f-472d-b95f-c68f050f0eef None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:08.438237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4661-4e8f-472d-b95f-c68f050f0eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:08.464070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-072b4661-4e8f-472d-b95f-c68f050f0eef None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:08.466875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-072b4661-4e8f-472d-b95f-c68f050f0eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:08.592984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e0902-89df-4126-a210-33b3984fb3fc None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:08.593459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e0902-89df-4126-a210-33b3984fb3fc None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:08.593546 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e0902-89df-4126-a210-33b3984fb3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:08.596634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 846/1727] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:08.633579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1e9e0902-89df-4126-a210-33b3984fb3fc None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:08.638033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1e9e0902-89df-4126-a210-33b3984fb3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:08.662955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 882/1728] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:08.668965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b4661-4e8f-472d-b95f-c68f050f0eef None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:08.683143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b4661-4e8f-472d-b95f-c68f050f0eef None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:08.694949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3fc52-4d72-4690-ac80-05d7b3fde772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:08.695605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3fc52-4d72-4690-ac80-05d7b3fde772 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:08.696089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3fc52-4d72-4690-ac80-05d7b3fde772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:08.696415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3fc52-4d72-4690-ac80-05d7b3fde772 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:08.697613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3fc52-4d72-4690-ac80-05d7b3fde772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:08.697613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3fc52-4d72-4690-ac80-05d7b3fde772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:08.702886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3fc52-4d72-4690-ac80-05d7b3fde772 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:08.741848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 847/1729] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:56:08.748759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e0902-89df-4126-a210-33b3984fb3fc None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:08.774602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e0902-89df-4126-a210-33b3984fb3fc None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:08.781611 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6c299-a48e-4ae8-9163-dcc223d6aadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:08.782797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6c299-a48e-4ae8-9163-dcc223d6aadc admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:08.783384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6c299-a48e-4ae8-9163-dcc223d6aadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:08.784615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6c299-a48e-4ae8-9163-dcc223d6aadc admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:08.784740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6c299-a48e-4ae8-9163-dcc223d6aadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:08.787069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6c299-a48e-4ae8-9163-dcc223d6aadc admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:08.793730 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3fc52-4d72-4690-ac80-05d7b3fde772 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:08.796532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7be3fc52-4d72-4690-ac80-05d7b3fde772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:08.815371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3fc52-4d72-4690-ac80-05d7b3fde772 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:08.821120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 883/1730] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:08 2022] POST /identity/v3/projects => generated 361 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:08.825489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6c299-a48e-4ae8-9163-dcc223d6aadc admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:08.825633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:08.827627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:08.828471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:08.828871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] PATH_INFO: `/v3/projects/288e179daad74a288313ef24d09b734b` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:08.830965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=288e179daad74a288313ef24d09b734b)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:08.833743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:08.887077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:08.928098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6c299-a48e-4ae8-9163-dcc223d6aadc admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:08.929143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '288e179daad74a288313ef24d09b734b', 'request_id': 'req-2dee690c-80e5-4b86-bde9-4b42724b2aee'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:08.937051 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '288e179daad74a288313ef24d09b734b', 'request_id': 'req-2dee690c-80e5-4b86-bde9-4b42724b2aee'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:08.945638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '288e179daad74a288313ef24d09b734b', 'request_id': 'req-2dee690c-80e5-4b86-bde9-4b42724b2aee'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:08.964118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] The token cache is being invalidate because project 288e179daad74a288313ef24d09b734b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:08.965669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:08.965669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:08.970471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 848/1731] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:08 2022] DELETE /identity/v3/projects/288e179daad74a288313ef24d09b734b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:08.971464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:08.974175 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:09.004152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06629a24-8380-43c0-9d89-1d190f8a5fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:09.008918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629a24-8380-43c0-9d89-1d190f8a5fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.009267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629a24-8380-43c0-9d89-1d190f8a5fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.009721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629a24-8380-43c0-9d89-1d190f8a5fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.010425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06629a24-8380-43c0-9d89-1d190f8a5fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:09.012463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06629a24-8380-43c0-9d89-1d190f8a5fd4 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:09.048533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:09.058079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 884/1732] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:08 2022] POST /identity/v3/users => generated 281 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:09.059007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06629a24-8380-43c0-9d89-1d190f8a5fd4 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:09.060805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-06629a24-8380-43c0-9d89-1d190f8a5fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:09.067147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee690c-80e5-4b86-bde9-4b42724b2aee admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:09.074869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06629a24-8380-43c0-9d89-1d190f8a5fd4 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:09.075809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:09.076720 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.077185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.077534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] PATH_INFO: `/v3/users/57322ee29b1c46d293f25ee905220234` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.078847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57322ee29b1c46d293f25ee905220234)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:09.081311 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e41cf-607b-45cf-bb13-0e02d1622c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:09.082086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e41cf-607b-45cf-bb13-0e02d1622c1e admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.082368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e41cf-607b-45cf-bb13-0e02d1622c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.082640 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e41cf-607b-45cf-bb13-0e02d1622c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.083301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e41cf-607b-45cf-bb13-0e02d1622c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:09.084952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e41cf-607b-45cf-bb13-0e02d1622c1e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:09.085053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:09.102264 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 885/1733] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:09.103974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e41cf-607b-45cf-bb13-0e02d1622c1e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:09.117571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e41cf-607b-45cf-bb13-0e02d1622c1e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:09.123294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf3a5a-dd0d-4a6e-860e-e12f21cd22b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:09.124522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3a5a-dd0d-4a6e-860e-e12f21cd22b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.124848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3a5a-dd0d-4a6e-860e-e12f21cd22b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.125258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3a5a-dd0d-4a6e-860e-e12f21cd22b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.126059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf3a5a-dd0d-4a6e-860e-e12f21cd22b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:09.127932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf3a5a-dd0d-4a6e-860e-e12f21cd22b3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:09.140344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 886/1734] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:09.144371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf3a5a-dd0d-4a6e-860e-e12f21cd22b3 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:09.162223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf3a5a-dd0d-4a6e-860e-e12f21cd22b3 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:09.164110 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57322ee29b1c46d293f25ee905220234'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.166077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57322ee29b1c46d293f25ee905220234'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.168402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356b41c-7ca3-43de-b0c4-58a8d4a88d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:09.169089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356b41c-7ca3-43de-b0c4-58a8d4a88d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.169359 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356b41c-7ca3-43de-b0c4-58a8d4a88d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.169683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356b41c-7ca3-43de-b0c4-58a8d4a88d4e admin admin] PATH_INFO: `/v3/projects/46c8cfa61fee433a81b3f06ff070739f/users/dfe93c33ed0a4a59b6191d463c445096/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.175581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57322ee29b1c46d293f25ee905220234'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.176477 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57322ee29b1c46d293f25ee905220234'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.195327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356b41c-7ca3-43de-b0c4-58a8d4a88d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c8cfa61fee433a81b3f06ff070739f, user_id=dfe93c33ed0a4a59b6191d463c445096, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:09.198668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57322ee29b1c46d293f25ee905220234'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.200067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356b41c-7ca3-43de-b0c4-58a8d4a88d4e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:09.227905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 887/1736] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:09 2022] PUT /identity/v3/projects/46c8cfa61fee433a81b3f06ff070739f/users/dfe93c33ed0a4a59b6191d463c445096/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:09.228202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 849/1736] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:08 2022] DELETE /identity/v3/users/57322ee29b1c46d293f25ee905220234 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:09.237587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e6959-8c76-46a8-b51d-bcc93b4c01bc None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.237872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e6959-8c76-46a8-b51d-bcc93b4c01bc None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.238143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e6959-8c76-46a8-b51d-bcc93b4c01bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.275490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-6f0e6959-8c76-46a8-b51d-bcc93b4c01bc None None] MFA Rules not processed for user `dfe93c33ed0a4a59b6191d463c445096`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:09.276939 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6f0e6959-8c76-46a8-b51d-bcc93b4c01bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:09.368386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 888/1737] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:09.407563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:09.409894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:09.477942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588107b7-4a9a-4529-9f85-57157e292254 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:09.485018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165cbdb-883a-4445-be5a-2accad9ac380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:09.486167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165cbdb-883a-4445-be5a-2accad9ac380 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.486167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165cbdb-883a-4445-be5a-2accad9ac380 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.486373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165cbdb-883a-4445-be5a-2accad9ac380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.497326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165cbdb-883a-4445-be5a-2accad9ac380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:09.499045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165cbdb-883a-4445-be5a-2accad9ac380 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:09.510912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 850/1738] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:09.530058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e6959-8c76-46a8-b51d-bcc93b4c01bc None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:09.531521 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6f0e6959-8c76-46a8-b51d-bcc93b4c01bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:09.562012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:09.625821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e6959-8c76-46a8-b51d-bcc93b4c01bc None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:09.631861 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:09.632776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.633714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.633989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] PATH_INFO: `/v3/users/dfe93c33ed0a4a59b6191d463c445096` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.635717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe93c33ed0a4a59b6191d463c445096)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:09.637867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:09.694200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165cbdb-883a-4445-be5a-2accad9ac380 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:09.696537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3165cbdb-883a-4445-be5a-2accad9ac380 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:09.728006 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe93c33ed0a4a59b6191d463c445096'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.728858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe93c33ed0a4a59b6191d463c445096'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.730187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165cbdb-883a-4445-be5a-2accad9ac380 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:09.736917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:09.737667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.737808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe93c33ed0a4a59b6191d463c445096'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.740954 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.741262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] PATH_INFO: `/v3/projects/95e8cc2243fb4b4ea501e7f8db711fc4` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.741665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe93c33ed0a4a59b6191d463c445096'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.743454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e8cc2243fb4b4ea501e7f8db711fc4)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:09.746042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:09.784922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-74e439ef-8a9d-43a8-895d-c4c976830469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe93c33ed0a4a59b6191d463c445096'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.814450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 889/1739] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:09 2022] DELETE /identity/v3/users/dfe93c33ed0a4a59b6191d463c445096 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:09.822167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93eaf4-09d2-4041-8e04-1f19dcc1896f None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.823393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93eaf4-09d2-4041-8e04-1f19dcc1896f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.823872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93eaf4-09d2-4041-8e04-1f19dcc1896f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.862294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7a93eaf4-09d2-4041-8e04-1f19dcc1896f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:09.869176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7a93eaf4-09d2-4041-8e04-1f19dcc1896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:09.869384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e8cc2243fb4b4ea501e7f8db711fc4', 'request_id': 'req-04970faa-a32f-4f59-8da9-595a0c5a66ac'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.886973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e8cc2243fb4b4ea501e7f8db711fc4', 'request_id': 'req-04970faa-a32f-4f59-8da9-595a0c5a66ac'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.898410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e8cc2243fb4b4ea501e7f8db711fc4', 'request_id': 'req-04970faa-a32f-4f59-8da9-595a0c5a66ac'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.901709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:09.929031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] The token cache is being invalidate because project 95e8cc2243fb4b4ea501e7f8db711fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:09.929494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.930584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04970faa-a32f-4f59-8da9-595a0c5a66ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:09.935770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 851/1740] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:09 2022] DELETE /identity/v3/projects/95e8cc2243fb4b4ea501e7f8db711fc4 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:09.937211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953764b6-8a7e-40ed-939a-448c03405465 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:09.937497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953764b6-8a7e-40ed-939a-448c03405465 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:09.937857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953764b6-8a7e-40ed-939a-448c03405465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:09.960722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-953764b6-8a7e-40ed-939a-448c03405465 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:09.962969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-953764b6-8a7e-40ed-939a-448c03405465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:10.027928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 890/1741] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:10.036523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93eaf4-09d2-4041-8e04-1f19dcc1896f None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:10.038410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7a93eaf4-09d2-4041-8e04-1f19dcc1896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:10.072937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 852/1742] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:10.080515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994b6cb-eb93-46b0-bae0-f0566bc7a73a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.080872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994b6cb-eb93-46b0-bae0-f0566bc7a73a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.081215 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994b6cb-eb93-46b0-bae0-f0566bc7a73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.134369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-7994b6cb-eb93-46b0-bae0-f0566bc7a73a None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:10.134369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7994b6cb-eb93-46b0-bae0-f0566bc7a73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:10.140303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:10.157373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 853/1743] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:10.162105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994b6cb-eb93-46b0-bae0-f0566bc7a73a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:10.170727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93eaf4-09d2-4041-8e04-1f19dcc1896f None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:10.177038 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf2a72-a97b-40f7-adfa-77ba84fa3aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:10.177430 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf2a72-a97b-40f7-adfa-77ba84fa3aba service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.177777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf2a72-a97b-40f7-adfa-77ba84fa3aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.178196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf2a72-a97b-40f7-adfa-77ba84fa3aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.186699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994b6cb-eb93-46b0-bae0-f0566bc7a73a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:10.194101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf2a72-a97b-40f7-adfa-77ba84fa3aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:10.194101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf2a72-a97b-40f7-adfa-77ba84fa3aba service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:10.204237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999ef92-cff0-49bb-8c1f-79e8681626c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:10.204858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999ef92-cff0-49bb-8c1f-79e8681626c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.205135 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999ef92-cff0-49bb-8c1f-79e8681626c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.205400 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999ef92-cff0-49bb-8c1f-79e8681626c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.206049 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999ef92-cff0-49bb-8c1f-79e8681626c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:10.206347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999ef92-cff0-49bb-8c1f-79e8681626c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:10.208220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999ef92-cff0-49bb-8c1f-79e8681626c2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:10.210135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 891/1744] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:56:10.225073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 854/1745] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:56:10.230126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf2a72-a97b-40f7-adfa-77ba84fa3aba service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:10.245390 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf2a72-a97b-40f7-adfa-77ba84fa3aba service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:10.256462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64630451-0d74-44ab-bfe5-ac69aafc2cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:10.257132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64630451-0d74-44ab-bfe5-ac69aafc2cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.257398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64630451-0d74-44ab-bfe5-ac69aafc2cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.257865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64630451-0d74-44ab-bfe5-ac69aafc2cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.259838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64630451-0d74-44ab-bfe5-ac69aafc2cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:10.262849 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64630451-0d74-44ab-bfe5-ac69aafc2cb7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:10.319618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 892/1746] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:10 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:10.324551 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999ef92-cff0-49bb-8c1f-79e8681626c2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:10.341216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999ef92-cff0-49bb-8c1f-79e8681626c2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:10.449662 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d8165-1045-4867-9678-11d71fbe0dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:10.450914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d8165-1045-4867-9678-11d71fbe0dcf admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.451541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d8165-1045-4867-9678-11d71fbe0dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.452087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d8165-1045-4867-9678-11d71fbe0dcf admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.453391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d8165-1045-4867-9678-11d71fbe0dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:10.456182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d8165-1045-4867-9678-11d71fbe0dcf admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:10.489157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64630451-0d74-44ab-bfe5-ac69aafc2cb7 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:10.497922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 855/1747] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:10 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:10.504308 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64630451-0d74-44ab-bfe5-ac69aafc2cb7 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:10.510963 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f5fc4-e164-4c63-8ff4-b8d06d506b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:10.512011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f5fc4-e164-4c63-8ff4-b8d06d506b61 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.512380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f5fc4-e164-4c63-8ff4-b8d06d506b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.512780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f5fc4-e164-4c63-8ff4-b8d06d506b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.513587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f5fc4-e164-4c63-8ff4-b8d06d506b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:10.515409 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f5fc4-e164-4c63-8ff4-b8d06d506b61 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:10.533967 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d8165-1045-4867-9678-11d71fbe0dcf admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:10.534496 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 893/1748] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:10.547906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d8165-1045-4867-9678-11d71fbe0dcf admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:10.554081 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2022d1-00a8-4fb8-87c1-72b0017c0cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:10.554706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2022d1-00a8-4fb8-87c1-72b0017c0cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.556901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2022d1-00a8-4fb8-87c1-72b0017c0cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.557266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2022d1-00a8-4fb8-87c1-72b0017c0cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.558041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2022d1-00a8-4fb8-87c1-72b0017c0cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:10.560051 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2022d1-00a8-4fb8-87c1-72b0017c0cb9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:10.573569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 856/1749] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:10.577408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f5fc4-e164-4c63-8ff4-b8d06d506b61 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:10.588954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f5fc4-e164-4c63-8ff4-b8d06d506b61 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:10.594210 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d452f-fd99-4ac8-9b33-ce6df8bc0df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:10.594877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d452f-fd99-4ac8-9b33-ce6df8bc0df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.595151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d452f-fd99-4ac8-9b33-ce6df8bc0df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.595473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d452f-fd99-4ac8-9b33-ce6df8bc0df1 admin admin] PATH_INFO: `/v3/projects/3e2b9be070c346a99cea9ef2b5c77740/users/3c1159ad642f4d80a552e2f2974c411d/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.624207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d452f-fd99-4ac8-9b33-ce6df8bc0df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2b9be070c346a99cea9ef2b5c77740, user_id=3c1159ad642f4d80a552e2f2974c411d, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:10.627238 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d452f-fd99-4ac8-9b33-ce6df8bc0df1 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:10.648000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2022d1-00a8-4fb8-87c1-72b0017c0cb9 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:10.650791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 894/1750] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:10 2022] PUT /identity/v3/projects/3e2b9be070c346a99cea9ef2b5c77740/users/3c1159ad642f4d80a552e2f2974c411d/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:10.651195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ab2022d1-00a8-4fb8-87c1-72b0017c0cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:10.662206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2955185-a643-4acb-8bbb-cdfcbc5703ab None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.662496 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2955185-a643-4acb-8bbb-cdfcbc5703ab None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.662770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2955185-a643-4acb-8bbb-cdfcbc5703ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.677533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2022d1-00a8-4fb8-87c1-72b0017c0cb9 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:10.699182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c2955185-a643-4acb-8bbb-cdfcbc5703ab None None] MFA Rules not processed for user `3c1159ad642f4d80a552e2f2974c411d`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:10.703271 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c2955185-a643-4acb-8bbb-cdfcbc5703ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:10.754555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:10.770745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:10.771906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.772198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.772477 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] PATH_INFO: `/v3/projects/46c8cfa61fee433a81b3f06ff070739f` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.774556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=46c8cfa61fee433a81b3f06ff070739f)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:10.776778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:10.816059 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 895/1751] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:10.828985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2955185-a643-4acb-8bbb-cdfcbc5703ab None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:10.830554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c2955185-a643-4acb-8bbb-cdfcbc5703ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:10.837064 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46c8cfa61fee433a81b3f06ff070739f', 'request_id': 'req-800d68a6-db83-47a2-a507-751d44101e1c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:10.843939 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46c8cfa61fee433a81b3f06ff070739f', 'request_id': 'req-800d68a6-db83-47a2-a507-751d44101e1c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:10.852915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46c8cfa61fee433a81b3f06ff070739f', 'request_id': 'req-800d68a6-db83-47a2-a507-751d44101e1c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:10.866330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] The token cache is being invalidate because project 46c8cfa61fee433a81b3f06ff070739f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:10.866854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:10.868481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:10.870173 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 857/1752] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:10 2022] DELETE /identity/v3/projects/46c8cfa61fee433a81b3f06ff070739f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:10.909922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2955185-a643-4acb-8bbb-cdfcbc5703ab None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:10.946294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:10.969639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f5380-3469-4e49-b87a-ae41feb215fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:10.970337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f5380-3469-4e49-b87a-ae41feb215fd service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:10.970620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f5380-3469-4e49-b87a-ae41feb215fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:10.970887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f5380-3469-4e49-b87a-ae41feb215fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:10.972836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-db1f5380-3469-4e49-b87a-ae41feb215fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:11.051770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f5380-3469-4e49-b87a-ae41feb215fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:11.053340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f5380-3469-4e49-b87a-ae41feb215fd service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:11.065513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 896/1753] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:56:12.973154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:12.974763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:13.044709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d68a6-db83-47a2-a507-751d44101e1c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:13.049601 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:13.050231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:13.050504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:13.050907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] PATH_INFO: `/v3/users/7706eb2365ab4a7f8b28711feae3c22f` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:13.052095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7706eb2365ab4a7f8b28711feae3c22f)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:13.054004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:13.100808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7706eb2365ab4a7f8b28711feae3c22f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.101397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7706eb2365ab4a7f8b28711feae3c22f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.110032 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7706eb2365ab4a7f8b28711feae3c22f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.110759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7706eb2365ab4a7f8b28711feae3c22f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.123804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7706eb2365ab4a7f8b28711feae3c22f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.136275 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 858/1754] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:12 2022] DELETE /identity/v3/users/7706eb2365ab4a7f8b28711feae3c22f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:13.335645 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f5380-3469-4e49-b87a-ae41feb215fd service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:13.351505 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f5380-3469-4e49-b87a-ae41feb215fd service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:13.422235 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:13.422810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:13.423085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:13.423426 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] PATH_INFO: `/v3/projects/813d1003a59e4dad9aa7764c2f1903ad` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:13.424538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] RBAC: Authorizing `identity:delete_project(project_id=813d1003a59e4dad9aa7764c2f1903ad)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:13.426467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:13.477860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '813d1003a59e4dad9aa7764c2f1903ad', 'request_id': 'req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.483084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '813d1003a59e4dad9aa7764c2f1903ad', 'request_id': 'req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.491131 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '813d1003a59e4dad9aa7764c2f1903ad', 'request_id': 'req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.503284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] The token cache is being invalidate because project 813d1003a59e4dad9aa7764c2f1903ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:13.503563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.504216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.505712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 897/1755] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:13 2022] DELETE /identity/v3/projects/813d1003a59e4dad9aa7764c2f1903ad => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:13.510582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:13.512192 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:13.520905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:13.529279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6a143-41ca-41fb-b833-3d35b8b08c83 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:13.626605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:13.627158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:13.627435 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:13.627709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] PATH_INFO: `/v3/users/7c2c0cc62ce44fbfae34d9b149ef134e` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:13.628872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2c0cc62ce44fbfae34d9b149ef134e)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:13.630785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:13.688639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:13.690088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:13.690297 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2c0cc62ce44fbfae34d9b149ef134e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.690943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2c0cc62ce44fbfae34d9b149ef134e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.699073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2c0cc62ce44fbfae34d9b149ef134e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.699735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2c0cc62ce44fbfae34d9b149ef134e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.703436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab17a2b-44c8-4c9f-af1c-b47121ed2faf admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:13.708839 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:13.709574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:13.710060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:13.710362 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] PATH_INFO: `/v3/users/3c1159ad642f4d80a552e2f2974c411d` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:13.713229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c1159ad642f4d80a552e2f2974c411d)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:13.714795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4b021f0b-ee1a-40aa-bb65-a3931f64d66d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2c0cc62ce44fbfae34d9b149ef134e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.715247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:13.729679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:13.735598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 859/1756] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:13 2022] DELETE /identity/v3/users/7c2c0cc62ce44fbfae34d9b149ef134e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:13.784261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c1159ad642f4d80a552e2f2974c411d'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.784931 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c1159ad642f4d80a552e2f2974c411d'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.795097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c1159ad642f4d80a552e2f2974c411d'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.795812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c1159ad642f4d80a552e2f2974c411d'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.812260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c1159ad642f4d80a552e2f2974c411d'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:13.828632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 898/1757] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:13 2022] DELETE /identity/v3/users/3c1159ad642f4d80a552e2f2974c411d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:13.833198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b8aa7-30b7-4511-9be8-d28e74a74c20 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:13.833551 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b8aa7-30b7-4511-9be8-d28e74a74c20 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:13.834250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b8aa7-30b7-4511-9be8-d28e74a74c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:13.857387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-231b8aa7-30b7-4511-9be8-d28e74a74c20 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:13.859373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-231b8aa7-30b7-4511-9be8-d28e74a74c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:13.931063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 860/1758] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:13.932415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:13.933832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:13.943095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b8aa7-30b7-4511-9be8-d28e74a74c20 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:13.948118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-231b8aa7-30b7-4511-9be8-d28e74a74c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:13.950109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c9b8e-e364-4067-9684-89306c881988 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:13.960166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:13.960823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:13.961208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:13.961907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] PATH_INFO: `/v3/projects/e6d45369785e4cd9b6acc04670162be5` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:13.963082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d45369785e4cd9b6acc04670162be5)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:13.965098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:13.984834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:14.031630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d45369785e4cd9b6acc04670162be5', 'request_id': 'req-2fc18360-3313-49f0-9eea-1e3db51438e1'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:14.039406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d45369785e4cd9b6acc04670162be5', 'request_id': 'req-2fc18360-3313-49f0-9eea-1e3db51438e1'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:14.043042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b8aa7-30b7-4511-9be8-d28e74a74c20 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.048065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d45369785e4cd9b6acc04670162be5', 'request_id': 'req-2fc18360-3313-49f0-9eea-1e3db51438e1'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:14.066193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] The token cache is being invalidate because project e6d45369785e4cd9b6acc04670162be5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:14.066564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:14.067297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2fc18360-3313-49f0-9eea-1e3db51438e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:14.069822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 899/1759] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:13 2022] DELETE /identity/v3/projects/e6d45369785e4cd9b6acc04670162be5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:14.080416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38b208-5425-45a7-8f47-6b2127d091ac None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.080416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38b208-5425-45a7-8f47-6b2127d091ac None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.080416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38b208-5425-45a7-8f47-6b2127d091ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.100204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2a38b208-5425-45a7-8f47-6b2127d091ac None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:14.101542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2a38b208-5425-45a7-8f47-6b2127d091ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:14.105329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b365875-3f72-4202-8d9f-274a542db37c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.105936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b365875-3f72-4202-8d9f-274a542db37c service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.106214 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b365875-3f72-4202-8d9f-274a542db37c service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.106481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b365875-3f72-4202-8d9f-274a542db37c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.108618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9b365875-3f72-4202-8d9f-274a542db37c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:14.167190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 900/1760] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:14.176283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c05b76-57fe-4af1-b948-7ecf9f5fdb5d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.176283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c05b76-57fe-4af1-b948-7ecf9f5fdb5d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.176283 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c05b76-57fe-4af1-b948-7ecf9f5fdb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.179079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b365875-3f72-4202-8d9f-274a542db37c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.181628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b365875-3f72-4202-8d9f-274a542db37c service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.191680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 861/1761] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:14.199548 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-20c05b76-57fe-4af1-b948-7ecf9f5fdb5d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:14.200988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-20c05b76-57fe-4af1-b948-7ecf9f5fdb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:14.209034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 901/1762] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:14.213395 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b365875-3f72-4202-8d9f-274a542db37c service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.223217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b365875-3f72-4202-8d9f-274a542db37c service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.231696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c96681-9df1-408d-86e6-1b3f3b132741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.232798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c96681-9df1-408d-86e6-1b3f3b132741 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.233074 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c96681-9df1-408d-86e6-1b3f3b132741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.233342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c96681-9df1-408d-86e6-1b3f3b132741 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.233990 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c96681-9df1-408d-86e6-1b3f3b132741 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:14.234287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c96681-9df1-408d-86e6-1b3f3b132741 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.235985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c96681-9df1-408d-86e6-1b3f3b132741 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.252125 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 862/1763] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:56:14.254143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c05b76-57fe-4af1-b948-7ecf9f5fdb5d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.260687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:14.264821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c05b76-57fe-4af1-b948-7ecf9f5fdb5d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.276794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfedeee-f60f-49ac-b840-fb727a63a3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.277697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfedeee-f60f-49ac-b840-fb727a63a3eb admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.278183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfedeee-f60f-49ac-b840-fb727a63a3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.278447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfedeee-f60f-49ac-b840-fb727a63a3eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.279138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfedeee-f60f-49ac-b840-fb727a63a3eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.281190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfedeee-f60f-49ac-b840-fb727a63a3eb admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.313138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 902/1764] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:14.316383 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c96681-9df1-408d-86e6-1b3f3b132741 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.328168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c96681-9df1-408d-86e6-1b3f3b132741 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.412453 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cabc38-61e5-47a1-a739-467f5091cd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.413063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cabc38-61e5-47a1-a739-467f5091cd7d admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.413337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cabc38-61e5-47a1-a739-467f5091cd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.413603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cabc38-61e5-47a1-a739-467f5091cd7d admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.415009 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cabc38-61e5-47a1-a739-467f5091cd7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.417070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cabc38-61e5-47a1-a739-467f5091cd7d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.443003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 863/1765] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/users => generated 291 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:14.444055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfedeee-f60f-49ac-b840-fb727a63a3eb admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.461478 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfedeee-f60f-49ac-b840-fb727a63a3eb admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.467848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2efcf-af34-430f-97d9-52fab0dc0c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.468644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2efcf-af34-430f-97d9-52fab0dc0c0d admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.468984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2efcf-af34-430f-97d9-52fab0dc0c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.469251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2efcf-af34-430f-97d9-52fab0dc0c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.469893 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2efcf-af34-430f-97d9-52fab0dc0c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.471492 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2efcf-af34-430f-97d9-52fab0dc0c0d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.483800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 903/1766] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:14.487810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cabc38-61e5-47a1-a739-467f5091cd7d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.495969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:14.500111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cabc38-61e5-47a1-a739-467f5091cd7d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.503028 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2efcf-af34-430f-97d9-52fab0dc0c0d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.504913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5ed2efcf-af34-430f-97d9-52fab0dc0c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:14.506343 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289151ac-eee4-45a8-9f5f-69f069904bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.506929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289151ac-eee4-45a8-9f5f-69f069904bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.507199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289151ac-eee4-45a8-9f5f-69f069904bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.507467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289151ac-eee4-45a8-9f5f-69f069904bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.508159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289151ac-eee4-45a8-9f5f-69f069904bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.510988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289151ac-eee4-45a8-9f5f-69f069904bb8 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.518466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2efcf-af34-430f-97d9-52fab0dc0c0d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.523726 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.524519 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.525034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.525350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] PATH_INFO: `/v3/projects/3e2b9be070c346a99cea9ef2b5c77740` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.526689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e2b9be070c346a99cea9ef2b5c77740)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.528771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.529870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 864/1767] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:14.535273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289151ac-eee4-45a8-9f5f-69f069904bb8 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.548216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289151ac-eee4-45a8-9f5f-69f069904bb8 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.553034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e247c6-1b70-447c-9723-3aab798e8625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.553682 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e247c6-1b70-447c-9723-3aab798e8625 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.553953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e247c6-1b70-447c-9723-3aab798e8625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.554223 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e247c6-1b70-447c-9723-3aab798e8625 admin admin] PATH_INFO: `/v3/projects/f1263257b3464ff582b539e724a99430/users/404db286fe0f4c0c8316d6c40f67eca3/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.572371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e247c6-1b70-447c-9723-3aab798e8625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1263257b3464ff582b539e724a99430, user_id=404db286fe0f4c0c8316d6c40f67eca3, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.575014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e247c6-1b70-447c-9723-3aab798e8625 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.581852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e2b9be070c346a99cea9ef2b5c77740', 'request_id': 'req-00cd2d0d-5100-493d-a933-0d2278e06917'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:14.588331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e2b9be070c346a99cea9ef2b5c77740', 'request_id': 'req-00cd2d0d-5100-493d-a933-0d2278e06917'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:14.588619 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 865/1768] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:14 2022] PUT /identity/v3/projects/f1263257b3464ff582b539e724a99430/users/404db286fe0f4c0c8316d6c40f67eca3/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:14.596826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e2b9be070c346a99cea9ef2b5c77740', 'request_id': 'req-00cd2d0d-5100-493d-a933-0d2278e06917'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:14.602100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8e0a2-9c92-4e06-a199-4bae3a61e960 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.604254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8e0a2-9c92-4e06-a199-4bae3a61e960 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.604587 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8e0a2-9c92-4e06-a199-4bae3a61e960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.613640 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] The token cache is being invalidate because project 3e2b9be070c346a99cea9ef2b5c77740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:14.613989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:14.614772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-00cd2d0d-5100-493d-a933-0d2278e06917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:14.616981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 904/1769] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:14 2022] DELETE /identity/v3/projects/3e2b9be070c346a99cea9ef2b5c77740 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:14.622289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a07094-53f2-47ad-99ca-a65eea1696b2 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.622636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a07094-53f2-47ad-99ca-a65eea1696b2 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.622969 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a07094-53f2-47ad-99ca-a65eea1696b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.639239 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-83a8e0a2-9c92-4e06-a199-4bae3a61e960 None None] MFA Rules not processed for user `404db286fe0f4c0c8316d6c40f67eca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:14.640491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-83a8e0a2-9c92-4e06-a199-4bae3a61e960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:14.645443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-d0a07094-53f2-47ad-99ca-a65eea1696b2 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:14.646684 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d0a07094-53f2-47ad-99ca-a65eea1696b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:14.726289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 905/1770] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:14.734323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f1db8-3116-497d-bdab-7425a835814d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.734610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f1db8-3116-497d-bdab-7425a835814d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.734883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f1db8-3116-497d-bdab-7425a835814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.735624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 866/1771] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:14.739839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8e0a2-9c92-4e06-a199-4bae3a61e960 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.741243 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-83a8e0a2-9c92-4e06-a199-4bae3a61e960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:14.756496 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-039f1db8-3116-497d-bdab-7425a835814d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:14.757402 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8e0a2-9c92-4e06-a199-4bae3a61e960 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.757703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-039f1db8-3116-497d-bdab-7425a835814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:14.765122 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84f679-ddde-4347-8fca-6613e8d239dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.765887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84f679-ddde-4347-8fca-6613e8d239dd admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.766258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84f679-ddde-4347-8fca-6613e8d239dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.766664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84f679-ddde-4347-8fca-6613e8d239dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.767391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84f679-ddde-4347-8fca-6613e8d239dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.769560 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84f679-ddde-4347-8fca-6613e8d239dd admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.772661 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 906/1772] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:14.779030 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f1db8-3116-497d-bdab-7425a835814d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.798924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f1db8-3116-497d-bdab-7425a835814d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.802083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 867/1773] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:14.804455 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaa560-19e9-49ff-8b0d-2dfbf42e7364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.805677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa560-19e9-49ff-8b0d-2dfbf42e7364 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.806023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa560-19e9-49ff-8b0d-2dfbf42e7364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.806365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa560-19e9-49ff-8b0d-2dfbf42e7364 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.807091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaa560-19e9-49ff-8b0d-2dfbf42e7364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:14.807396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaa560-19e9-49ff-8b0d-2dfbf42e7364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.807770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84f679-ddde-4347-8fca-6613e8d239dd admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.809170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaa560-19e9-49ff-8b0d-2dfbf42e7364 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.821898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 907/1774] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:56:14.826128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaa560-19e9-49ff-8b0d-2dfbf42e7364 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.835737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaa560-19e9-49ff-8b0d-2dfbf42e7364 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.872039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84f679-ddde-4347-8fca-6613e8d239dd admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.872256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:14.876865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde14a1-a1f4-4962-914a-0bed6e4ab318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.877471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde14a1-a1f4-4962-914a-0bed6e4ab318 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.877751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde14a1-a1f4-4962-914a-0bed6e4ab318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.878020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde14a1-a1f4-4962-914a-0bed6e4ab318 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.878666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde14a1-a1f4-4962-914a-0bed6e4ab318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.880649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde14a1-a1f4-4962-914a-0bed6e4ab318 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.901687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5112b8-959e-419b-8b83-71663a7ac637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.902288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5112b8-959e-419b-8b83-71663a7ac637 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.902417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 868/1775] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:14.904172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5112b8-959e-419b-8b83-71663a7ac637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.904536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5112b8-959e-419b-8b83-71663a7ac637 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.905183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5112b8-959e-419b-8b83-71663a7ac637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.906285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde14a1-a1f4-4962-914a-0bed6e4ab318 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.907135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5112b8-959e-419b-8b83-71663a7ac637 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.918753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde14a1-a1f4-4962-914a-0bed6e4ab318 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.923671 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62c3d6-ac5f-4d62-986a-69cfd1ed8554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:14.924345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62c3d6-ac5f-4d62-986a-69cfd1ed8554 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:14.924617 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62c3d6-ac5f-4d62-986a-69cfd1ed8554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:14.924886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62c3d6-ac5f-4d62-986a-69cfd1ed8554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:14.925521 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62c3d6-ac5f-4d62-986a-69cfd1ed8554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:14.927227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62c3d6-ac5f-4d62-986a-69cfd1ed8554 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:14.939167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 869/1776] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:14.940595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 908/1777] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:14.943545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62c3d6-ac5f-4d62-986a-69cfd1ed8554 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.944763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5112b8-959e-419b-8b83-71663a7ac637 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:14.956696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62c3d6-ac5f-4d62-986a-69cfd1ed8554 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:14.969451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:15.021055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5112b8-959e-419b-8b83-71663a7ac637 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.026684 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f3328-5647-427e-946a-cca215cbe075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.027298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f3328-5647-427e-946a-cca215cbe075 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.027548 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37448796-2014-48bd-83de-b9745faf981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.028023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f3328-5647-427e-946a-cca215cbe075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.028023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f3328-5647-427e-946a-cca215cbe075 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.028215 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37448796-2014-48bd-83de-b9745faf981f admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.028461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37448796-2014-48bd-83de-b9745faf981f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.028552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f3328-5647-427e-946a-cca215cbe075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.028843 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37448796-2014-48bd-83de-b9745faf981f admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.029550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37448796-2014-48bd-83de-b9745faf981f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.030539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f3328-5647-427e-946a-cca215cbe075 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.031183 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37448796-2014-48bd-83de-b9745faf981f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.046193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 870/1778] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:15.048798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37448796-2014-48bd-83de-b9745faf981f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.059405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 909/1779] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:14 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:15.062851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37448796-2014-48bd-83de-b9745faf981f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.063610 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f3328-5647-427e-946a-cca215cbe075 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.067834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2041ca-eed9-4078-b7c9-c66fb816b78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.068526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2041ca-eed9-4078-b7c9-c66fb816b78c admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.068796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2041ca-eed9-4078-b7c9-c66fb816b78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.069061 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2041ca-eed9-4078-b7c9-c66fb816b78c admin admin] PATH_INFO: `/v3/projects/deecb4f93dca40bea8b9d3f0c546a2bd/users/0ddaf79e15c041489b92177448149cd1/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.079200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f3328-5647-427e-946a-cca215cbe075 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.084184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1e40a-45f9-4fe1-8f9c-608c91036981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.084804 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1e40a-45f9-4fe1-8f9c-608c91036981 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.085085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1e40a-45f9-4fe1-8f9c-608c91036981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.085419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1e40a-45f9-4fe1-8f9c-608c91036981 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.086014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1e40a-45f9-4fe1-8f9c-608c91036981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.087681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2041ca-eed9-4078-b7c9-c66fb816b78c admin admin] RBAC: Authorizing `identity:create_grant(project_id=deecb4f93dca40bea8b9d3f0c546a2bd, user_id=0ddaf79e15c041489b92177448149cd1, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.087800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1e40a-45f9-4fe1-8f9c-608c91036981 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.090518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2041ca-eed9-4078-b7c9-c66fb816b78c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.099735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 910/1780] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:15.103630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1e40a-45f9-4fe1-8f9c-608c91036981 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.115766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1e40a-45f9-4fe1-8f9c-608c91036981 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.119702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 871/1781] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:15 2022] PUT /identity/v3/projects/deecb4f93dca40bea8b9d3f0c546a2bd/users/0ddaf79e15c041489b92177448149cd1/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:15.129671 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259807e-d403-48aa-b014-5b29646f71b9 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.129948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259807e-d403-48aa-b014-5b29646f71b9 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.130244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259807e-d403-48aa-b014-5b29646f71b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.164728 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-3259807e-d403-48aa-b014-5b29646f71b9 None None] MFA Rules not processed for user `0ddaf79e15c041489b92177448149cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:15.166623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3259807e-d403-48aa-b014-5b29646f71b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:15.181428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc0c2a-6c2c-4281-9c3c-600283a669f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.182033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0c2a-6c2c-4281-9c3c-600283a669f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.182302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0c2a-6c2c-4281-9c3c-600283a669f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.182571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0c2a-6c2c-4281-9c3c-600283a669f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.183209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc0c2a-6c2c-4281-9c3c-600283a669f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.184837 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc0c2a-6c2c-4281-9c3c-600283a669f7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.194812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 911/1782] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:15.198799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc0c2a-6c2c-4281-9c3c-600283a669f7 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.208549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc0c2a-6c2c-4281-9c3c-600283a669f7 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.215544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297eb99-7994-404b-b067-6245db7048e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.216855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297eb99-7994-404b-b067-6245db7048e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.217138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297eb99-7994-404b-b067-6245db7048e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.217413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297eb99-7994-404b-b067-6245db7048e8 admin admin] PATH_INFO: `/v3/projects/ba1f788091014fe390ce7f7b0d8e0a5a/users/158efa641285434589e232c4d4429e56/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.234591 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297eb99-7994-404b-b067-6245db7048e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba1f788091014fe390ce7f7b0d8e0a5a, user_id=158efa641285434589e232c4d4429e56, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.234758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 872/1783] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:15.237432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297eb99-7994-404b-b067-6245db7048e8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.239149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259807e-d403-48aa-b014-5b29646f71b9 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.251070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259807e-d403-48aa-b014-5b29646f71b9 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.251563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 912/1784] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:15 2022] PUT /identity/v3/projects/ba1f788091014fe390ce7f7b0d8e0a5a/users/158efa641285434589e232c4d4429e56/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:15.262365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f3858-7207-4c19-a5b0-eea667c9d5cf None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.262793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f3858-7207-4c19-a5b0-eea667c9d5cf None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.263066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f3858-7207-4c19-a5b0-eea667c9d5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.284671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:15.294592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9b7f3858-7207-4c19-a5b0-eea667c9d5cf None None] MFA Rules not processed for user `158efa641285434589e232c4d4429e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:15.296295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9b7f3858-7207-4c19-a5b0-eea667c9d5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:15.325553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda5ce4-1465-44fd-b2e6-09f9d4d55cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.326163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda5ce4-1465-44fd-b2e6-09f9d4d55cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.326438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda5ce4-1465-44fd-b2e6-09f9d4d55cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.326707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda5ce4-1465-44fd-b2e6-09f9d4d55cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.327334 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda5ce4-1465-44fd-b2e6-09f9d4d55cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.329447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda5ce4-1465-44fd-b2e6-09f9d4d55cc3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.366612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 873/1785] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:15 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:15.366842 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 913/1786] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:15.369002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda5ce4-1465-44fd-b2e6-09f9d4d55cc3 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.373213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:15.377927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f3858-7207-4c19-a5b0-eea667c9d5cf None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.379293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9b7f3858-7207-4c19-a5b0-eea667c9d5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:15.457350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda5ce4-1465-44fd-b2e6-09f9d4d55cc3 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.458515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f3858-7207-4c19-a5b0-eea667c9d5cf None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.463358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bb672-e97b-4bcf-ac22-80ce049d5a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.464010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb672-e97b-4bcf-ac22-80ce049d5a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.464354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb672-e97b-4bcf-ac22-80ce049d5a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.464685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb672-e97b-4bcf-ac22-80ce049d5a92 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.465383 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bb672-e97b-4bcf-ac22-80ce049d5a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.467350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bb672-e97b-4bcf-ac22-80ce049d5a92 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.496647 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 874/1787] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:15 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:15.497969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bb672-e97b-4bcf-ac22-80ce049d5a92 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.508874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bb672-e97b-4bcf-ac22-80ce049d5a92 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.514123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef12f47-89d5-4950-a4c2-c7859407b989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.514754 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef12f47-89d5-4950-a4c2-c7859407b989 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.515031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef12f47-89d5-4950-a4c2-c7859407b989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.515303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef12f47-89d5-4950-a4c2-c7859407b989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.515975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef12f47-89d5-4950-a4c2-c7859407b989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.517571 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef12f47-89d5-4950-a4c2-c7859407b989 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.517922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130f42-642c-4858-b9f3-8c80c541abe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.518528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130f42-642c-4858-b9f3-8c80c541abe7 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.518883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130f42-642c-4858-b9f3-8c80c541abe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.519071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130f42-642c-4858-b9f3-8c80c541abe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.529456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 875/1788] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:15.533483 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef12f47-89d5-4950-a4c2-c7859407b989 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.543756 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef12f47-89d5-4950-a4c2-c7859407b989 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.551100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41172b-d84b-4ff1-b47c-b6024a7e0cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.551100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41172b-d84b-4ff1-b47c-b6024a7e0cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.551100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41172b-d84b-4ff1-b47c-b6024a7e0cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.551100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41172b-d84b-4ff1-b47c-b6024a7e0cb6 admin admin] PATH_INFO: `/v3/projects/4e665fff22f542fabc8d72fd3dd3ba9c/users/3ade18ee7dff4ee2991aa544567aad23/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.565054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41172b-d84b-4ff1-b47c-b6024a7e0cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e665fff22f542fabc8d72fd3dd3ba9c, user_id=3ade18ee7dff4ee2991aa544567aad23, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.567771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41172b-d84b-4ff1-b47c-b6024a7e0cb6 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.581616 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 876/1789] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:15 2022] PUT /identity/v3/projects/4e665fff22f542fabc8d72fd3dd3ba9c/users/3ade18ee7dff4ee2991aa544567aad23/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:15.583866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e130f42-642c-4858-b9f3-8c80c541abe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.585348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41172b-d84b-4ff1-b47c-b6024a7e0cb6 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.585492 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e130f42-642c-4858-b9f3-8c80c541abe7 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.597946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 914/1790] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:56:15.624137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130f42-642c-4858-b9f3-8c80c541abe7 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.635627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130f42-642c-4858-b9f3-8c80c541abe7 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.644020 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:15.655850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41172b-d84b-4ff1-b47c-b6024a7e0cb6 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.661144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88074c60-498b-42af-ab1c-3b10f91db7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.662153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88074c60-498b-42af-ab1c-3b10f91db7fa admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.662477 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88074c60-498b-42af-ab1c-3b10f91db7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.662833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88074c60-498b-42af-ab1c-3b10f91db7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.663573 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88074c60-498b-42af-ab1c-3b10f91db7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.666628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88074c60-498b-42af-ab1c-3b10f91db7fa admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.678635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 877/1791] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:15.683403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88074c60-498b-42af-ab1c-3b10f91db7fa admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.694271 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88074c60-498b-42af-ab1c-3b10f91db7fa admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.699838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1583-f0b4-4477-b761-28284b9f52fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.700439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.700656 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1583-f0b4-4477-b761-28284b9f52fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.700936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1583-f0b4-4477-b761-28284b9f52fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.701064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.701209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1583-f0b4-4477-b761-28284b9f52fe admin admin] PATH_INFO: `/v3/projects/4e665fff22f542fabc8d72fd3dd3ba9c/users/3ade18ee7dff4ee2991aa544567aad23/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.701341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.701619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] PATH_INFO: `/v3/users/158efa641285434589e232c4d4429e56` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.702829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=158efa641285434589e232c4d4429e56)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.703231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1583-f0b4-4477-b761-28284b9f52fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e665fff22f542fabc8d72fd3dd3ba9c, user_id=3ade18ee7dff4ee2991aa544567aad23, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:15.704797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.705899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1583-f0b4-4477-b761-28284b9f52fe admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:15.716894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 878/1792] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:15 2022] PUT /identity/v3/projects/4e665fff22f542fabc8d72fd3dd3ba9c/users/3ade18ee7dff4ee2991aa544567aad23/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:15.727444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d66ff-4a09-463c-a502-81f596d66ea0 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.727821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d66ff-4a09-463c-a502-81f596d66ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.728125 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d66ff-4a09-463c-a502-81f596d66ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.759171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-935d66ff-4a09-463c-a502-81f596d66ea0 None None] MFA Rules not processed for user `3ade18ee7dff4ee2991aa544567aad23`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:15.759465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '158efa641285434589e232c4d4429e56'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:15.760209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '158efa641285434589e232c4d4429e56'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:15.760418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-935d66ff-4a09-463c-a502-81f596d66ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:15.767475 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '158efa641285434589e232c4d4429e56'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:15.768214 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '158efa641285434589e232c4d4429e56'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:15.784739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3388c62f-d020-4a23-bf19-6fc6b5a9ec31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '158efa641285434589e232c4d4429e56'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:15.792214 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:15.799428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 915/1793] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:15 2022] DELETE /identity/v3/users/158efa641285434589e232c4d4429e56 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:15.804058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229878d5-caba-4d36-824c-92414c333ee2 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.804358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229878d5-caba-4d36-824c-92414c333ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.806310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229878d5-caba-4d36-824c-92414c333ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.831488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-229878d5-caba-4d36-824c-92414c333ee2 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:15.832747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-229878d5-caba-4d36-824c-92414c333ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:15.841244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 879/1794] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:15.853283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d66ff-4a09-463c-a502-81f596d66ea0 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.855086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-935d66ff-4a09-463c-a502-81f596d66ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:15.902717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 916/1795] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:15.912775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229878d5-caba-4d36-824c-92414c333ee2 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:15.914212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-229878d5-caba-4d36-824c-92414c333ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:15.932738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d66ff-4a09-463c-a502-81f596d66ea0 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.955447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:15.991905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229878d5-caba-4d36-824c-92414c333ee2 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:15.994267 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cdb02-941c-4398-802a-764e7ea6aa59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.994940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cdb02-941c-4398-802a-764e7ea6aa59 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.995156 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cdb02-941c-4398-802a-764e7ea6aa59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.995426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cdb02-941c-4398-802a-764e7ea6aa59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:15.996239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f11463-e328-4297-b50f-44544cfb2ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:15.996842 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f11463-e328-4297-b50f-44544cfb2ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:15.997116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f11463-e328-4297-b50f-44544cfb2ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:15.997352 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f69cdb02-941c-4398-802a-764e7ea6aa59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:15.997462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f11463-e328-4297-b50f-44544cfb2ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:16.007009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f11463-e328-4297-b50f-44544cfb2ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:16.008695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f11463-e328-4297-b50f-44544cfb2ec5 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:16.023697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 917/1796] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:16.084172 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cdb02-941c-4398-802a-764e7ea6aa59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:16.085678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cdb02-941c-4398-802a-764e7ea6aa59 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:16.097006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 880/1797] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:56:16.401647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f11463-e328-4297-b50f-44544cfb2ec5 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:16.405245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b7f11463-e328-4297-b50f-44544cfb2ec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:16.430204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f11463-e328-4297-b50f-44544cfb2ec5 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:16.443636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:16.444316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:16.444597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:16.444873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] PATH_INFO: `/v3/projects/ba1f788091014fe390ce7f7b0d8e0a5a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:16.446046 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba1f788091014fe390ce7f7b0d8e0a5a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:16.448854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:16.548090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba1f788091014fe390ce7f7b0d8e0a5a', 'request_id': 'req-66159559-c73f-47e6-9c04-ae4c4a5ff295'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:16.555843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba1f788091014fe390ce7f7b0d8e0a5a', 'request_id': 'req-66159559-c73f-47e6-9c04-ae4c4a5ff295'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:16.564848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba1f788091014fe390ce7f7b0d8e0a5a', 'request_id': 'req-66159559-c73f-47e6-9c04-ae4c4a5ff295'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:16.578381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] The token cache is being invalidate because project ba1f788091014fe390ce7f7b0d8e0a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:16.578761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:16.579427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:16.580984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 918/1798] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:16 2022] DELETE /identity/v3/projects/ba1f788091014fe390ce7f7b0d8e0a5a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:17.854776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cdb02-941c-4398-802a-764e7ea6aa59 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:17.863653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cdb02-941c-4398-802a-764e7ea6aa59 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:17.904274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:17.923912 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c4f3e-3016-4ab2-9ef2-4dfe8b8261fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:17.924545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c4f3e-3016-4ab2-9ef2-4dfe8b8261fc service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:17.924820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c4f3e-3016-4ab2-9ef2-4dfe8b8261fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:17.925087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c4f3e-3016-4ab2-9ef2-4dfe8b8261fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:17.926929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3a4c4f3e-3016-4ab2-9ef2-4dfe8b8261fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:17.988345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c4f3e-3016-4ab2-9ef2-4dfe8b8261fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:17.989849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c4f3e-3016-4ab2-9ef2-4dfe8b8261fc service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:18.000790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 881/1799] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:18.552089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:18.553536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:18.566604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66159559-c73f-47e6-9c04-ae4c4a5ff295 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:18.571418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ee56f-e14e-4aba-91b3-6de4189ce00f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:18.572054 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ee56f-e14e-4aba-91b3-6de4189ce00f service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:18.572331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ee56f-e14e-4aba-91b3-6de4189ce00f service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:18.572600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ee56f-e14e-4aba-91b3-6de4189ce00f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:18.574494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-938ee56f-e14e-4aba-91b3-6de4189ce00f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:18.641592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ee56f-e14e-4aba-91b3-6de4189ce00f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:18.643041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ee56f-e14e-4aba-91b3-6de4189ce00f service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:18.653332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 919/1800] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:22.695462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c4f3e-3016-4ab2-9ef2-4dfe8b8261fc service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:22.700680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3a4c4f3e-3016-4ab2-9ef2-4dfe8b8261fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:22.792607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c4f3e-3016-4ab2-9ef2-4dfe8b8261fc service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:22.800556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:22.801198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:22.801522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:22.801790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] PATH_INFO: `/v3/users/404db286fe0f4c0c8316d6c40f67eca3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:22.803072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=404db286fe0f4c0c8316d6c40f67eca3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:22.805360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:22.868859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404db286fe0f4c0c8316d6c40f67eca3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:22.870539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404db286fe0f4c0c8316d6c40f67eca3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:22.882646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404db286fe0f4c0c8316d6c40f67eca3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:22.883784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404db286fe0f4c0c8316d6c40f67eca3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:22.903240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404db286fe0f4c0c8316d6c40f67eca3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:22.918572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 882/1801] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:22 2022] DELETE /identity/v3/users/404db286fe0f4c0c8316d6c40f67eca3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:22.924293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30f8f2-e588-4da8-a04a-e6d2f002ffeb None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:22.924757 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30f8f2-e588-4da8-a04a-e6d2f002ffeb None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:22.924883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30f8f2-e588-4da8-a04a-e6d2f002ffeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:22.950646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ae30f8f2-e588-4da8-a04a-e6d2f002ffeb None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:22.952154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ae30f8f2-e588-4da8-a04a-e6d2f002ffeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:23.008084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:23.030839 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 920/1802] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:23.042486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:23.044092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:23.056087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:23.115396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af056e8d-e9ca-4c3c-b33c-3114cbb975e6 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:23.123283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f147a-bd50-4132-9894-95a053d39492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:23.124052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f147a-bd50-4132-9894-95a053d39492 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:23.124339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f147a-bd50-4132-9894-95a053d39492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:23.124763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f147a-bd50-4132-9894-95a053d39492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:23.136629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f147a-bd50-4132-9894-95a053d39492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:23.138185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f147a-bd50-4132-9894-95a053d39492 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:23.147934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 883/1803] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:23.704544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30f8f2-e588-4da8-a04a-e6d2f002ffeb None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:23.706133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ae30f8f2-e588-4da8-a04a-e6d2f002ffeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:23.719674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30f8f2-e588-4da8-a04a-e6d2f002ffeb None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:23.725792 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:23.726797 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f147a-bd50-4132-9894-95a053d39492 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:23.728310 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0b5f147a-bd50-4132-9894-95a053d39492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:23.730437 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:23.731503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:23.731784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] PATH_INFO: `/v3/users/07441f3b8a3841b0ba3677ec38fe7f32` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:23.733114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07441f3b8a3841b0ba3677ec38fe7f32)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:23.735121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:23.742692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f147a-bd50-4132-9894-95a053d39492 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:23.748830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:23.749565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:23.750532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:23.750532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] PATH_INFO: `/v3/projects/f1263257b3464ff582b539e724a99430` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:23.752067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1263257b3464ff582b539e724a99430)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:23.754200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:23.794717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07441f3b8a3841b0ba3677ec38fe7f32'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:23.795441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07441f3b8a3841b0ba3677ec38fe7f32'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:23.803247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07441f3b8a3841b0ba3677ec38fe7f32'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:23.804280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07441f3b8a3841b0ba3677ec38fe7f32'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:23.814865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1263257b3464ff582b539e724a99430', 'request_id': 'req-313cc917-28dd-4f01-a537-c500eefe4f26'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:23.820631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1263257b3464ff582b539e724a99430', 'request_id': 'req-313cc917-28dd-4f01-a537-c500eefe4f26'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:23.828953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1263257b3464ff582b539e724a99430', 'request_id': 'req-313cc917-28dd-4f01-a537-c500eefe4f26'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:23.833224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07441f3b8a3841b0ba3677ec38fe7f32'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:23.842890 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] The token cache is being invalidate because project f1263257b3464ff582b539e724a99430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:23.843171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:23.843800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:23.845355 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 884/1804] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:23 2022] DELETE /identity/v3/projects/f1263257b3464ff582b539e724a99430 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:23.849176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 921/1805] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:23 2022] DELETE /identity/v3/users/07441f3b8a3841b0ba3677ec38fe7f32 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:23.849407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:23.850617 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:23.920594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cc917-28dd-4f01-a537-c500eefe4f26 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:23.968338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:23.986710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:23.987344 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:23.987619 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:23.987904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] PATH_INFO: `/v3/users/0ddaf79e15c041489b92177448149cd1` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:23.989048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ddaf79e15c041489b92177448149cd1)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:23.990976 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.037828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.039633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:24.045920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ddaf79e15c041489b92177448149cd1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.046699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ddaf79e15c041489b92177448149cd1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.056195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ddaf79e15c041489b92177448149cd1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.057806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ddaf79e15c041489b92177448149cd1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.061745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871a64f-e398-4c1c-be2e-459611f21f48 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:24.070920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:24.072127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.072954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.073112 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] PATH_INFO: `/v3/projects/bfe90c27e0e44c4794ccff00eac150ca` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.073626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-24105d0a-fad2-4040-82a9-875aa4055004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ddaf79e15c041489b92177448149cd1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.074395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe90c27e0e44c4794ccff00eac150ca)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:24.078643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.093807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 885/1806] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:23 2022] DELETE /identity/v3/users/0ddaf79e15c041489b92177448149cd1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:24.142708 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe90c27e0e44c4794ccff00eac150ca', 'request_id': 'req-7c50e8df-a723-4696-b118-6e4e73192cb3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.149019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe90c27e0e44c4794ccff00eac150ca', 'request_id': 'req-7c50e8df-a723-4696-b118-6e4e73192cb3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.158458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe90c27e0e44c4794ccff00eac150ca', 'request_id': 'req-7c50e8df-a723-4696-b118-6e4e73192cb3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.174621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] The token cache is being invalidate because project bfe90c27e0e44c4794ccff00eac150ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:24.174959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.175692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7c50e8df-a723-4696-b118-6e4e73192cb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.177284 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 922/1807] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:24 2022] DELETE /identity/v3/projects/bfe90c27e0e44c4794ccff00eac150ca => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:24.182379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c65ecc-e634-4950-b1df-dd1c46575e3b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.182596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c65ecc-e634-4950-b1df-dd1c46575e3b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.182746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c65ecc-e634-4950-b1df-dd1c46575e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.209219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-b2c65ecc-e634-4950-b1df-dd1c46575e3b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:24.209696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b2c65ecc-e634-4950-b1df-dd1c46575e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:24.282940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 886/1808] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:24.290742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45a5dc-8d4f-424b-ad4a-cffe4c644ba0 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.291652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45a5dc-8d4f-424b-ad4a-cffe4c644ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.291947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45a5dc-8d4f-424b-ad4a-cffe4c644ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.313585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0b45a5dc-8d4f-424b-ad4a-cffe4c644ba0 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:24.314789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0b45a5dc-8d4f-424b-ad4a-cffe4c644ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:24.323211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 923/1809] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:24.327607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c65ecc-e634-4950-b1df-dd1c46575e3b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.348822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c65ecc-e634-4950-b1df-dd1c46575e3b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:24.359569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810618c7-8434-4e8b-ba03-f924b0f04940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:24.360251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810618c7-8434-4e8b-ba03-f924b0f04940 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.360531 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810618c7-8434-4e8b-ba03-f924b0f04940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.360805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810618c7-8434-4e8b-ba03-f924b0f04940 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.361457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810618c7-8434-4e8b-ba03-f924b0f04940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:24.361758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810618c7-8434-4e8b-ba03-f924b0f04940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:24.363360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810618c7-8434-4e8b-ba03-f924b0f04940 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.377743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 887/1810] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:56:24.381898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45a5dc-8d4f-424b-ad4a-cffe4c644ba0 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.396110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45a5dc-8d4f-424b-ad4a-cffe4c644ba0 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:24.401804 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6745f6-3d3a-4c2a-9b6c-31465a2b2a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:24.402485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6745f6-3d3a-4c2a-9b6c-31465a2b2a1f admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.402793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6745f6-3d3a-4c2a-9b6c-31465a2b2a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.403063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6745f6-3d3a-4c2a-9b6c-31465a2b2a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.403863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6745f6-3d3a-4c2a-9b6c-31465a2b2a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:24.406005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6745f6-3d3a-4c2a-9b6c-31465a2b2a1f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.413709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810618c7-8434-4e8b-ba03-f924b0f04940 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.415582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-810618c7-8434-4e8b-ba03-f924b0f04940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:24.430248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810618c7-8434-4e8b-ba03-f924b0f04940 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:24.436439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:24.437159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.437530 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.437868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] PATH_INFO: `/v3/projects/deecb4f93dca40bea8b9d3f0c546a2bd` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.439093 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 924/1811] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:24 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:24.440005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] RBAC: Authorizing `identity:delete_project(project_id=deecb4f93dca40bea8b9d3f0c546a2bd)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:24.443392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.443486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6745f6-3d3a-4c2a-9b6c-31465a2b2a1f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.464060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:24.489925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:24.494098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deecb4f93dca40bea8b9d3f0c546a2bd', 'request_id': 'req-f938bf2e-5f77-4f09-b25e-9d317423bfff'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.501357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deecb4f93dca40bea8b9d3f0c546a2bd', 'request_id': 'req-f938bf2e-5f77-4f09-b25e-9d317423bfff'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.508925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deecb4f93dca40bea8b9d3f0c546a2bd', 'request_id': 'req-f938bf2e-5f77-4f09-b25e-9d317423bfff'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.511840 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6745f6-3d3a-4c2a-9b6c-31465a2b2a1f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:24.521973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] The token cache is being invalidate because project deecb4f93dca40bea8b9d3f0c546a2bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:24.522261 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.522916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.524650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 888/1812] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:24 2022] DELETE /identity/v3/projects/deecb4f93dca40bea8b9d3f0c546a2bd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:24.528985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.530383 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:24.569606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6ec47-6de0-4d6e-aee0-9f89f625b344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:24.570240 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6ec47-6de0-4d6e-aee0-9f89f625b344 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.570515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6ec47-6de0-4d6e-aee0-9f89f625b344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.570785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6ec47-6de0-4d6e-aee0-9f89f625b344 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.571413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6ec47-6de0-4d6e-aee0-9f89f625b344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:24.573816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6ec47-6de0-4d6e-aee0-9f89f625b344 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.595920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 925/1813] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:24 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:24.597826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938bf2e-5f77-4f09-b25e-9d317423bfff admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:24.599325 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6ec47-6de0-4d6e-aee0-9f89f625b344 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.600921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ebe6ec47-6de0-4d6e-aee0-9f89f625b344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:24.602574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:24.603200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.603570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.604937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] PATH_INFO: `/v3/users/3ade18ee7dff4ee2991aa544567aad23` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.606066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ade18ee7dff4ee2991aa544567aad23)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:24.608000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.613245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6ec47-6de0-4d6e-aee0-9f89f625b344 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:24.617988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab31477-efd2-4750-8097-fd1c90f68040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:24.618749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab31477-efd2-4750-8097-fd1c90f68040 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.619027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab31477-efd2-4750-8097-fd1c90f68040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.619413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab31477-efd2-4750-8097-fd1c90f68040 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.620128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab31477-efd2-4750-8097-fd1c90f68040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:24.621706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab31477-efd2-4750-8097-fd1c90f68040 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.631752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 926/1814] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:24.636179 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab31477-efd2-4750-8097-fd1c90f68040 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.646033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab31477-efd2-4750-8097-fd1c90f68040 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:24.651338 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d41a8-6301-41aa-a1bf-b7a700c7c9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:24.651946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d41a8-6301-41aa-a1bf-b7a700c7c9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.652222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d41a8-6301-41aa-a1bf-b7a700c7c9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.652532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d41a8-6301-41aa-a1bf-b7a700c7c9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.653110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d41a8-6301-41aa-a1bf-b7a700c7c9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:24.654713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d41a8-6301-41aa-a1bf-b7a700c7c9cb admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.659964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ade18ee7dff4ee2991aa544567aad23'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.660698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ade18ee7dff4ee2991aa544567aad23'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.665460 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 927/1815] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:24.670311 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d41a8-6301-41aa-a1bf-b7a700c7c9cb admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.670712 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ade18ee7dff4ee2991aa544567aad23'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.671449 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ade18ee7dff4ee2991aa544567aad23'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.680195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d41a8-6301-41aa-a1bf-b7a700c7c9cb admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:24.683616 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e76e6be4-295f-4746-bee9-716351623515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ade18ee7dff4ee2991aa544567aad23'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:24.685648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711f459-9801-43be-93ca-925fcf4265a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:24.687190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711f459-9801-43be-93ca-925fcf4265a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.687470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711f459-9801-43be-93ca-925fcf4265a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.687745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711f459-9801-43be-93ca-925fcf4265a7 admin admin] PATH_INFO: `/v3/projects/c6416b6136b245708e5117844027e10e/users/f1256ec7650944b89ebd90eaeced4c3d/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.696645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 889/1816] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:24 2022] DELETE /identity/v3/users/3ade18ee7dff4ee2991aa544567aad23 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:24.709594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711f459-9801-43be-93ca-925fcf4265a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6416b6136b245708e5117844027e10e, user_id=f1256ec7650944b89ebd90eaeced4c3d, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:24.712415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711f459-9801-43be-93ca-925fcf4265a7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.726725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 928/1817] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:24 2022] PUT /identity/v3/projects/c6416b6136b245708e5117844027e10e/users/f1256ec7650944b89ebd90eaeced4c3d/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:24.737847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e42eea-02c9-4350-9a03-3005eda1f28b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.738418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e42eea-02c9-4350-9a03-3005eda1f28b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.738418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e42eea-02c9-4350-9a03-3005eda1f28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.768991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-04e42eea-02c9-4350-9a03-3005eda1f28b None None] MFA Rules not processed for user `f1256ec7650944b89ebd90eaeced4c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:24.770665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-04e42eea-02c9-4350-9a03-3005eda1f28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:24.815115 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:24.856003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 890/1818] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:24.861115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711f459-9801-43be-93ca-925fcf4265a7 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.863023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b711f459-9801-43be-93ca-925fcf4265a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:24.930869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711f459-9801-43be-93ca-925fcf4265a7 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:24.935643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a26cb5-ef18-4deb-86b2-6ea0e4f403e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:24.936285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a26cb5-ef18-4deb-86b2-6ea0e4f403e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:24.936564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a26cb5-ef18-4deb-86b2-6ea0e4f403e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:24.936835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a26cb5-ef18-4deb-86b2-6ea0e4f403e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:24.937467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a26cb5-ef18-4deb-86b2-6ea0e4f403e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:24.939411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a26cb5-ef18-4deb-86b2-6ea0e4f403e1 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:24.954754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:24.973125 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 929/1819] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:24 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:24.974363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e42eea-02c9-4350-9a03-3005eda1f28b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:24.986113 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e42eea-02c9-4350-9a03-3005eda1f28b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.065188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a26cb5-ef18-4deb-86b2-6ea0e4f403e1 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.065714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd79d1-fb5f-4674-9381-9f09ec54eeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.066571 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd79d1-fb5f-4674-9381-9f09ec54eeff admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.066783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd79d1-fb5f-4674-9381-9f09ec54eeff admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.067055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd79d1-fb5f-4674-9381-9f09ec54eeff admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.067872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b0a26cb5-ef18-4deb-86b2-6ea0e4f403e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:25.068010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd79d1-fb5f-4674-9381-9f09ec54eeff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.069834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd79d1-fb5f-4674-9381-9f09ec54eeff admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.087398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a26cb5-ef18-4deb-86b2-6ea0e4f403e1 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.093115 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 891/1820] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:24 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:25.097069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.097349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd79d1-fb5f-4674-9381-9f09ec54eeff admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.097718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.098061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.098315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] PATH_INFO: `/v3/projects/4e665fff22f542fabc8d72fd3dd3ba9c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.099442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e665fff22f542fabc8d72fd3dd3ba9c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.101422 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.107987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd79d1-fb5f-4674-9381-9f09ec54eeff admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.114087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ccd9f-4a5b-44ef-a697-5b526320db61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.114726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ccd9f-4a5b-44ef-a697-5b526320db61 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.115030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ccd9f-4a5b-44ef-a697-5b526320db61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.115332 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ccd9f-4a5b-44ef-a697-5b526320db61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.116374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ccd9f-4a5b-44ef-a697-5b526320db61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.118397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ccd9f-4a5b-44ef-a697-5b526320db61 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.128816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 892/1821] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:25.132547 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ccd9f-4a5b-44ef-a697-5b526320db61 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.143791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ccd9f-4a5b-44ef-a697-5b526320db61 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.154732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e665fff22f542fabc8d72fd3dd3ba9c', 'request_id': 'req-57c38ef6-3668-4955-b11d-b29ada8555e3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:25.160650 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e665fff22f542fabc8d72fd3dd3ba9c', 'request_id': 'req-57c38ef6-3668-4955-b11d-b29ada8555e3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:25.169440 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e665fff22f542fabc8d72fd3dd3ba9c', 'request_id': 'req-57c38ef6-3668-4955-b11d-b29ada8555e3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:25.181670 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] The token cache is being invalidate because project 4e665fff22f542fabc8d72fd3dd3ba9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:25.182042 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:25.182729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-57c38ef6-3668-4955-b11d-b29ada8555e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:25.184463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 930/1822] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:25 2022] DELETE /identity/v3/projects/4e665fff22f542fabc8d72fd3dd3ba9c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:25.189039 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270adb-41ab-44cd-82c0-112d2b7d7404 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.189318 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270adb-41ab-44cd-82c0-112d2b7d7404 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.189589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270adb-41ab-44cd-82c0-112d2b7d7404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.202404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0955e4-5a1e-4411-b5fc-8d45e733923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.203058 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0955e4-5a1e-4411-b5fc-8d45e733923d admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.203335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0955e4-5a1e-4411-b5fc-8d45e733923d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.203601 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0955e4-5a1e-4411-b5fc-8d45e733923d admin admin] PATH_INFO: `/v3/projects/2e119a670f1847eab6ba8372bf4833fd/users/36e1f0e1dfe24101aed76fe1eb731d59/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.209149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-92270adb-41ab-44cd-82c0-112d2b7d7404 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:25.210350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-92270adb-41ab-44cd-82c0-112d2b7d7404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:25.218452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 931/1823] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:25.218738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0955e4-5a1e-4411-b5fc-8d45e733923d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e119a670f1847eab6ba8372bf4833fd, user_id=36e1f0e1dfe24101aed76fe1eb731d59, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.221291 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0955e4-5a1e-4411-b5fc-8d45e733923d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.226183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381dac5-1492-455d-8fd2-2f52d50aad99 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.226569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381dac5-1492-455d-8fd2-2f52d50aad99 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.226873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381dac5-1492-455d-8fd2-2f52d50aad99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.235639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 893/1824] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:25 2022] PUT /identity/v3/projects/2e119a670f1847eab6ba8372bf4833fd/users/36e1f0e1dfe24101aed76fe1eb731d59/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:25.239415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0955e4-5a1e-4411-b5fc-8d45e733923d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.240983 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-aa0955e4-5a1e-4411-b5fc-8d45e733923d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:25.248231 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8381dac5-1492-455d-8fd2-2f52d50aad99 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:25.249425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8381dac5-1492-455d-8fd2-2f52d50aad99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:25.287708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:25.307956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0955e4-5a1e-4411-b5fc-8d45e733923d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.311929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 932/1825] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:25.312570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751f00e-a55a-407a-b296-00b1182a8e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.313219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751f00e-a55a-407a-b296-00b1182a8e96 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.313499 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751f00e-a55a-407a-b296-00b1182a8e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.313767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751f00e-a55a-407a-b296-00b1182a8e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.314408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751f00e-a55a-407a-b296-00b1182a8e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.316026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751f00e-a55a-407a-b296-00b1182a8e96 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.316211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381dac5-1492-455d-8fd2-2f52d50aad99 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.325270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381dac5-1492-455d-8fd2-2f52d50aad99 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.329458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 894/1826] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:25.330239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4ba79-0ebf-4169-8baf-a889549bd4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.330382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751f00e-a55a-407a-b296-00b1182a8e96 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.331086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4ba79-0ebf-4169-8baf-a889549bd4fd admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.331363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4ba79-0ebf-4169-8baf-a889549bd4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.331630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4ba79-0ebf-4169-8baf-a889549bd4fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.332435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4ba79-0ebf-4169-8baf-a889549bd4fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:25.332733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4ba79-0ebf-4169-8baf-a889549bd4fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.334485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4ba79-0ebf-4169-8baf-a889549bd4fd admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.340606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751f00e-a55a-407a-b296-00b1182a8e96 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.345302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a7544-5d02-426b-81b5-5ccc298b380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.345943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a7544-5d02-426b-81b5-5ccc298b380c admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.346222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a7544-5d02-426b-81b5-5ccc298b380c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.346518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a7544-5d02-426b-81b5-5ccc298b380c admin admin] PATH_INFO: `/v3/projects/2e119a670f1847eab6ba8372bf4833fd/users/36e1f0e1dfe24101aed76fe1eb731d59/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.348443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a7544-5d02-426b-81b5-5ccc298b380c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e119a670f1847eab6ba8372bf4833fd, user_id=36e1f0e1dfe24101aed76fe1eb731d59, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.351066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a7544-5d02-426b-81b5-5ccc298b380c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.354128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 933/1827] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:56:25.358411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4ba79-0ebf-4169-8baf-a889549bd4fd admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.362649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 895/1828] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:25 2022] PUT /identity/v3/projects/2e119a670f1847eab6ba8372bf4833fd/users/36e1f0e1dfe24101aed76fe1eb731d59/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:25.368336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4ba79-0ebf-4169-8baf-a889549bd4fd admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.373817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05819c3a-8a22-4a69-b8e2-d1f71ab07d4d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.374142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05819c3a-8a22-4a69-b8e2-d1f71ab07d4d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.374438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05819c3a-8a22-4a69-b8e2-d1f71ab07d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.406946 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-05819c3a-8a22-4a69-b8e2-d1f71ab07d4d None None] MFA Rules not processed for user `36e1f0e1dfe24101aed76fe1eb731d59`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:25.407700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-05819c3a-8a22-4a69-b8e2-d1f71ab07d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:25.421801 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:25.429926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd8351-4146-4682-ad4b-284c33f0916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.430696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd8351-4146-4682-ad4b-284c33f0916d admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.430981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd8351-4146-4682-ad4b-284c33f0916d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.431485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd8351-4146-4682-ad4b-284c33f0916d admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.433033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd8351-4146-4682-ad4b-284c33f0916d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.435408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd8351-4146-4682-ad4b-284c33f0916d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.467736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 934/1829] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:25 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:25.471948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd8351-4146-4682-ad4b-284c33f0916d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.486358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 896/1830] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:25.498378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05819c3a-8a22-4a69-b8e2-d1f71ab07d4d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.499800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-05819c3a-8a22-4a69-b8e2-d1f71ab07d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:25.533731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd8351-4146-4682-ad4b-284c33f0916d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.539549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65ab9c-9ac3-4b7b-9b9c-5c5f090718c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.540175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65ab9c-9ac3-4b7b-9b9c-5c5f090718c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.540449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65ab9c-9ac3-4b7b-9b9c-5c5f090718c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.540716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65ab9c-9ac3-4b7b-9b9c-5c5f090718c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.541372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65ab9c-9ac3-4b7b-9b9c-5c5f090718c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.543406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65ab9c-9ac3-4b7b-9b9c-5c5f090718c0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.558371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05819c3a-8a22-4a69-b8e2-d1f71ab07d4d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.566255 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 935/1831] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:25 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:25.570411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65ab9c-9ac3-4b7b-9b9c-5c5f090718c0 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.580378 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65ab9c-9ac3-4b7b-9b9c-5c5f090718c0 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.584961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe3558-1e79-400c-a8c3-c4f99eba7924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.585543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe3558-1e79-400c-a8c3-c4f99eba7924 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.585814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe3558-1e79-400c-a8c3-c4f99eba7924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.586079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe3558-1e79-400c-a8c3-c4f99eba7924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.586706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe3558-1e79-400c-a8c3-c4f99eba7924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.589152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe3558-1e79-400c-a8c3-c4f99eba7924 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.599188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 936/1832] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:25.602877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe3558-1e79-400c-a8c3-c4f99eba7924 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.608732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcfae2-a6b7-4fa3-8801-b8b12295eba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.609336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcfae2-a6b7-4fa3-8801-b8b12295eba6 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.609610 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcfae2-a6b7-4fa3-8801-b8b12295eba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.609879 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcfae2-a6b7-4fa3-8801-b8b12295eba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.613342 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe3558-1e79-400c-a8c3-c4f99eba7924 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.618073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6368792-2880-4bce-82f3-898299a9e8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.618658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6368792-2880-4bce-82f3-898299a9e8fd admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.618933 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6368792-2880-4bce-82f3-898299a9e8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.619202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6368792-2880-4bce-82f3-898299a9e8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.619833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6368792-2880-4bce-82f3-898299a9e8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.621428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6368792-2880-4bce-82f3-898299a9e8fd admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.628724 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:25.632480 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 937/1833] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:25.637373 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6368792-2880-4bce-82f3-898299a9e8fd admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.647268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6368792-2880-4bce-82f3-898299a9e8fd admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.652626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffc7ef-fa27-4247-86d2-036f7c5280a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.653262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc7ef-fa27-4247-86d2-036f7c5280a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.653540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc7ef-fa27-4247-86d2-036f7c5280a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.653811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc7ef-fa27-4247-86d2-036f7c5280a7 admin admin] PATH_INFO: `/v3/projects/e9854190f10741bc8636cec4604943d5/users/f676b3649fd04f699a5c5758fdc94017/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.669225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffc7ef-fa27-4247-86d2-036f7c5280a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9854190f10741bc8636cec4604943d5, user_id=f676b3649fd04f699a5c5758fdc94017, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.671952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffc7ef-fa27-4247-86d2-036f7c5280a7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.677410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bcfae2-a6b7-4fa3-8801-b8b12295eba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.678998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bcfae2-a6b7-4fa3-8801-b8b12295eba6 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.686106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 938/1834] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:25 2022] PUT /identity/v3/projects/e9854190f10741bc8636cec4604943d5/users/f676b3649fd04f699a5c5758fdc94017/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:25.689683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 897/1835] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:56:25.696445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c717c-dc5d-4bc4-871a-8f54ceadcdea None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.696728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c717c-dc5d-4bc4-871a-8f54ceadcdea None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.697029 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c717c-dc5d-4bc4-871a-8f54ceadcdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.726660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-bb4c717c-dc5d-4bc4-871a-8f54ceadcdea None None] MFA Rules not processed for user `f676b3649fd04f699a5c5758fdc94017`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:25.727904 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-bb4c717c-dc5d-4bc4-871a-8f54ceadcdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:25.738192 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcfae2-a6b7-4fa3-8801-b8b12295eba6 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.739825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-73bcfae2-a6b7-4fa3-8801-b8b12295eba6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:25.753015 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcfae2-a6b7-4fa3-8801-b8b12295eba6 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.793590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 939/1836] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:25.804152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c717c-dc5d-4bc4-871a-8f54ceadcdea None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:25.805134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458fe1e-7da8-4b97-9458-e42c003f5545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.805739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458fe1e-7da8-4b97-9458-e42c003f5545 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.806015 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458fe1e-7da8-4b97-9458-e42c003f5545 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.806282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458fe1e-7da8-4b97-9458-e42c003f5545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.808256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0458fe1e-7da8-4b97-9458-e42c003f5545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:25.813559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c717c-dc5d-4bc4-871a-8f54ceadcdea None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:25.817620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9713b-59b7-4d8b-920b-609e820ee1e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:25.818220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9713b-59b7-4d8b-920b-609e820ee1e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:25.818816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9713b-59b7-4d8b-920b-609e820ee1e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:25.818816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9713b-59b7-4d8b-920b-609e820ee1e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:25.827556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9713b-59b7-4d8b-920b-609e820ee1e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.829036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9713b-59b7-4d8b-920b-609e820ee1e7 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.837776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 940/1837] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:56:25.869663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458fe1e-7da8-4b97-9458-e42c003f5545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:25.871191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458fe1e-7da8-4b97-9458-e42c003f5545 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:25.881493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 898/1838] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:56:27.219044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ea83a-c4f8-425d-9a54-f6c75b7611b5 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:27.219525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ea83a-c4f8-425d-9a54-f6c75b7611b5 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:27.219637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ea83a-c4f8-425d-9a54-f6c75b7611b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:27.226567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:27.241588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-013ea83a-c4f8-425d-9a54-f6c75b7611b5 None None] MFA Rules not processed for user `f676b3649fd04f699a5c5758fdc94017`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:27.242800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-013ea83a-c4f8-425d-9a54-f6c75b7611b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:27.250403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 941/1839] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:29.453420 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458fe1e-7da8-4b97-9458-e42c003f5545 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:29.525201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458fe1e-7da8-4b97-9458-e42c003f5545 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:29.530646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:29.531353 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:29.531706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:29.532394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] PATH_INFO: `/v3/users/f1256ec7650944b89ebd90eaeced4c3d` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:29.533312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1256ec7650944b89ebd90eaeced4c3d)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:29.535660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:29.549005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:29.586262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1256ec7650944b89ebd90eaeced4c3d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:29.586915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1256ec7650944b89ebd90eaeced4c3d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:29.595451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1256ec7650944b89ebd90eaeced4c3d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:29.596159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1256ec7650944b89ebd90eaeced4c3d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:29.609902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1256ec7650944b89ebd90eaeced4c3d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:29.623797 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 899/1840] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:29 2022] DELETE /identity/v3/users/f1256ec7650944b89ebd90eaeced4c3d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:29.628431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058105-968a-45c2-a2a5-77d27bb8ec52 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:29.628957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058105-968a-45c2-a2a5-77d27bb8ec52 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:29.629053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058105-968a-45c2-a2a5-77d27bb8ec52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:29.649833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-8f058105-968a-45c2-a2a5-77d27bb8ec52 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:29.651629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8f058105-968a-45c2-a2a5-77d27bb8ec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:29.712852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 942/1841] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:29.723276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:29.724715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:29.784154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ac82b-9161-43bd-9f52-07355eaf55cb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:29.787960 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d33678-8eb2-44f1-9026-de8f5dc60bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:29.788563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d33678-8eb2-44f1-9026-de8f5dc60bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:29.788837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d33678-8eb2-44f1-9026-de8f5dc60bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:29.789106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d33678-8eb2-44f1-9026-de8f5dc60bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:29.798162 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d33678-8eb2-44f1-9026-de8f5dc60bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:29.799591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d33678-8eb2-44f1-9026-de8f5dc60bf7 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:29.809871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 900/1842] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:30.529327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f058105-968a-45c2-a2a5-77d27bb8ec52 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:30.530874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8f058105-968a-45c2-a2a5-77d27bb8ec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:30.545959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f058105-968a-45c2-a2a5-77d27bb8ec52 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:30.562050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:30.562733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:30.562947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:30.563214 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] PATH_INFO: `/v3/projects/c6416b6136b245708e5117844027e10e` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:30.564431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6416b6136b245708e5117844027e10e)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:30.566409 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:30.613290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6416b6136b245708e5117844027e10e', 'request_id': 'req-430f8cff-6554-4d15-825a-4972bbb2f69c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:30.619029 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6416b6136b245708e5117844027e10e', 'request_id': 'req-430f8cff-6554-4d15-825a-4972bbb2f69c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:30.626524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6416b6136b245708e5117844027e10e', 'request_id': 'req-430f8cff-6554-4d15-825a-4972bbb2f69c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:30.638246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] The token cache is being invalidate because project c6416b6136b245708e5117844027e10e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:30.638841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:30.639141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:30.640649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 943/1843] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:30 2022] DELETE /identity/v3/projects/c6416b6136b245708e5117844027e10e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:30.644794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d33678-8eb2-44f1-9026-de8f5dc60bf7 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:30.654588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d33678-8eb2-44f1-9026-de8f5dc60bf7 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:30.711563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:30.712225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:30.712504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:30.712777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] PATH_INFO: `/v3/users/36e1f0e1dfe24101aed76fe1eb731d59` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:30.713922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e1f0e1dfe24101aed76fe1eb731d59)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:30.715828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:30.753487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:30.764962 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e1f0e1dfe24101aed76fe1eb731d59'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:30.765621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e1f0e1dfe24101aed76fe1eb731d59'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:30.772657 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e1f0e1dfe24101aed76fe1eb731d59'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:30.773374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e1f0e1dfe24101aed76fe1eb731d59'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:30.785861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0275508e-26e4-4ef4-8dd1-aa15e3b88a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e1f0e1dfe24101aed76fe1eb731d59'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:30.799023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 901/1844] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:30 2022] DELETE /identity/v3/users/36e1f0e1dfe24101aed76fe1eb731d59 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:31.063782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:31.065278 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:31.080688 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f8cff-6554-4d15-825a-4972bbb2f69c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:31.140836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:31.141472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.141748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.142021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] PATH_INFO: `/v3/projects/2e119a670f1847eab6ba8372bf4833fd` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.143143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e119a670f1847eab6ba8372bf4833fd)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:31.145086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:31.169151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:31.195136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e119a670f1847eab6ba8372bf4833fd', 'request_id': 'req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:31.201071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e119a670f1847eab6ba8372bf4833fd', 'request_id': 'req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:31.209126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e119a670f1847eab6ba8372bf4833fd', 'request_id': 'req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:31.221503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] The token cache is being invalidate because project 2e119a670f1847eab6ba8372bf4833fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:31.221794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:31.222504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53f26a20-0f74-480f-8599-e0c5bcfc2b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:31.224212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 944/1845] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:31 2022] DELETE /identity/v3/projects/2e119a670f1847eab6ba8372bf4833fd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:31.229521 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c04d1-d91d-49ff-ab37-88f993752e46 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.229798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c04d1-d91d-49ff-ab37-88f993752e46 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.230071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c04d1-d91d-49ff-ab37-88f993752e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.250251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f88c04d1-d91d-49ff-ab37-88f993752e46 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:31.251578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f88c04d1-d91d-49ff-ab37-88f993752e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:31.310746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 902/1846] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:31.318611 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5ca0-32a7-482a-aefe-e0c34b33b82b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.318815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5ca0-32a7-482a-aefe-e0c34b33b82b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.319089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5ca0-32a7-482a-aefe-e0c34b33b82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.339097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f14a5ca0-32a7-482a-aefe-e0c34b33b82b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:31.340332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f14a5ca0-32a7-482a-aefe-e0c34b33b82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:31.348500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 945/1847] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:31.352801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c04d1-d91d-49ff-ab37-88f993752e46 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:31.362990 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c04d1-d91d-49ff-ab37-88f993752e46 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:31.367538 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9f2b6-0056-44ed-812c-81b3217c86fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:31.368153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9f2b6-0056-44ed-812c-81b3217c86fd admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.368425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9f2b6-0056-44ed-812c-81b3217c86fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.368689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9f2b6-0056-44ed-812c-81b3217c86fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.369352 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9f2b6-0056-44ed-812c-81b3217c86fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:31.369642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9f2b6-0056-44ed-812c-81b3217c86fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:31.371324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9f2b6-0056-44ed-812c-81b3217c86fd admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:31.382515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 903/1848] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:56:31.386916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a5ca0-32a7-482a-aefe-e0c34b33b82b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:31.396126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a5ca0-32a7-482a-aefe-e0c34b33b82b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:31.401305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe34e1-8e7d-4c4f-b5f7-c549ea399066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:31.401899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe34e1-8e7d-4c4f-b5f7-c549ea399066 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.402167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe34e1-8e7d-4c4f-b5f7-c549ea399066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.402431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe34e1-8e7d-4c4f-b5f7-c549ea399066 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.403088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe34e1-8e7d-4c4f-b5f7-c549ea399066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:31.405025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe34e1-8e7d-4c4f-b5f7-c549ea399066 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:31.432433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 946/1849] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:31 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:31.436831 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9f2b6-0056-44ed-812c-81b3217c86fd admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:31.446684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9f2b6-0056-44ed-812c-81b3217c86fd admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:31.503626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594fe01-dd66-4d66-bc81-68cf12b2fab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:31.504280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594fe01-dd66-4d66-bc81-68cf12b2fab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.504552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594fe01-dd66-4d66-bc81-68cf12b2fab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.504820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594fe01-dd66-4d66-bc81-68cf12b2fab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.505446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594fe01-dd66-4d66-bc81-68cf12b2fab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:31.507336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594fe01-dd66-4d66-bc81-68cf12b2fab6 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:31.534721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 904/1850] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:31 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:31.536133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe34e1-8e7d-4c4f-b5f7-c549ea399066 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:31.546828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe34e1-8e7d-4c4f-b5f7-c549ea399066 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:31.551391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13df97-fda4-4142-88d7-41b24dc5b3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:31.552016 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13df97-fda4-4142-88d7-41b24dc5b3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.552287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13df97-fda4-4142-88d7-41b24dc5b3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.552552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13df97-fda4-4142-88d7-41b24dc5b3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.553184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13df97-fda4-4142-88d7-41b24dc5b3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:31.554781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13df97-fda4-4142-88d7-41b24dc5b3f0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:31.568543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 947/1851] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:31.573581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594fe01-dd66-4d66-bc81-68cf12b2fab6 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:31.584090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594fe01-dd66-4d66-bc81-68cf12b2fab6 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:31.589048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17720a9-2217-4061-ac00-fcf042dd0d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:31.589678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17720a9-2217-4061-ac00-fcf042dd0d9e admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.589980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17720a9-2217-4061-ac00-fcf042dd0d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.590248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17720a9-2217-4061-ac00-fcf042dd0d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.590886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17720a9-2217-4061-ac00-fcf042dd0d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:31.592499 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17720a9-2217-4061-ac00-fcf042dd0d9e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:31.603648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 905/1852] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:31.608002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13df97-fda4-4142-88d7-41b24dc5b3f0 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:31.619701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13df97-fda4-4142-88d7-41b24dc5b3f0 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:31.624759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73091490-d6e0-4911-9068-53540ddb145a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:31.625476 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73091490-d6e0-4911-9068-53540ddb145a admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.625765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73091490-d6e0-4911-9068-53540ddb145a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.626036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73091490-d6e0-4911-9068-53540ddb145a admin admin] PATH_INFO: `/v3/projects/93ebc5636b0840c5960a9fd265c8f55a/users/5395aaca2a81470ba45f7075c256ec68/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.641989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73091490-d6e0-4911-9068-53540ddb145a admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ebc5636b0840c5960a9fd265c8f55a, user_id=5395aaca2a81470ba45f7075c256ec68, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:31.644688 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73091490-d6e0-4911-9068-53540ddb145a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:31.657970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 948/1853] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:31 2022] PUT /identity/v3/projects/93ebc5636b0840c5960a9fd265c8f55a/users/5395aaca2a81470ba45f7075c256ec68/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:31.670660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0eed4-ed41-4d38-a891-a798d3c8d6e8 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.671217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0eed4-ed41-4d38-a891-a798d3c8d6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.671217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0eed4-ed41-4d38-a891-a798d3c8d6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.703128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-7ad0eed4-ed41-4d38-a891-a798d3c8d6e8 None None] MFA Rules not processed for user `5395aaca2a81470ba45f7075c256ec68`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:31.704452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7ad0eed4-ed41-4d38-a891-a798d3c8d6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:31.709625 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:31.773699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 906/1854] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:31.778139 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73091490-d6e0-4911-9068-53540ddb145a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:31.846636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73091490-d6e0-4911-9068-53540ddb145a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:31.851381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be390-a1c4-4aba-ba92-42be9e12b93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:31.852196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be390-a1c4-4aba-ba92-42be9e12b93d admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.852470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be390-a1c4-4aba-ba92-42be9e12b93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.852739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be390-a1c4-4aba-ba92-42be9e12b93d admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.853414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96be390-a1c4-4aba-ba92-42be9e12b93d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:31.855978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96be390-a1c4-4aba-ba92-42be9e12b93d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:31.883795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 949/1855] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:31 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:31.888048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0eed4-ed41-4d38-a891-a798d3c8d6e8 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:31.899107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0eed4-ed41-4d38-a891-a798d3c8d6e8 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:31.957739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04031ecf-5cc7-480c-a017-954f528fa5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:31.958344 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04031ecf-5cc7-480c-a017-954f528fa5c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:31.958658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04031ecf-5cc7-480c-a017-954f528fa5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:31.958928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04031ecf-5cc7-480c-a017-954f528fa5c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:31.959565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04031ecf-5cc7-480c-a017-954f528fa5c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:31.961544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04031ecf-5cc7-480c-a017-954f528fa5c9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:31.984255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 907/1856] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:31 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:31.988267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be390-a1c4-4aba-ba92-42be9e12b93d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:31.999084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be390-a1c4-4aba-ba92-42be9e12b93d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:32.003695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ef74f-93e0-4771-b79c-f686bde8100a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:32.004332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ef74f-93e0-4771-b79c-f686bde8100a admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:32.004606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ef74f-93e0-4771-b79c-f686bde8100a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:32.004872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ef74f-93e0-4771-b79c-f686bde8100a admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:32.005568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ef74f-93e0-4771-b79c-f686bde8100a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:32.007167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ef74f-93e0-4771-b79c-f686bde8100a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:32.021168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 950/1857] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:32.022369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04031ecf-5cc7-480c-a017-954f528fa5c9 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:32.033252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04031ecf-5cc7-480c-a017-954f528fa5c9 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:32.039148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb078c-504b-44f3-bf9b-989ada78d12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:32.039808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb078c-504b-44f3-bf9b-989ada78d12e admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:32.040133 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb078c-504b-44f3-bf9b-989ada78d12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:32.040404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb078c-504b-44f3-bf9b-989ada78d12e admin admin] PATH_INFO: `/v3/projects/ad64f2f20d3543429bec366a6034022c/users/e70c356a5992411189e31ced50c5868c/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:32.057488 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb078c-504b-44f3-bf9b-989ada78d12e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad64f2f20d3543429bec366a6034022c, user_id=e70c356a5992411189e31ced50c5868c, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:32.060531 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb078c-504b-44f3-bf9b-989ada78d12e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:32.075650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 908/1858] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:32 2022] PUT /identity/v3/projects/ad64f2f20d3543429bec366a6034022c/users/e70c356a5992411189e31ced50c5868c/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:32.079648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ef74f-93e0-4771-b79c-f686bde8100a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:32.090777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ef74f-93e0-4771-b79c-f686bde8100a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:32.108938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:32.152261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f386369-02bb-472e-9dce-f9a1026726e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:32.152875 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386369-02bb-472e-9dce-f9a1026726e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:32.153149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386369-02bb-472e-9dce-f9a1026726e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:32.153417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386369-02bb-472e-9dce-f9a1026726e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:32.154056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f386369-02bb-472e-9dce-f9a1026726e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:32.155669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f386369-02bb-472e-9dce-f9a1026726e0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:32.166025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 951/1859] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:32.170909 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb078c-504b-44f3-bf9b-989ada78d12e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:32.181474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb078c-504b-44f3-bf9b-989ada78d12e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:32.186272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063825d-2434-41b7-aa69-944bad92ab00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:32.186922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063825d-2434-41b7-aa69-944bad92ab00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:32.187197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063825d-2434-41b7-aa69-944bad92ab00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:32.187468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063825d-2434-41b7-aa69-944bad92ab00 admin admin] PATH_INFO: `/v3/projects/ad64f2f20d3543429bec366a6034022c/users/e70c356a5992411189e31ced50c5868c/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:32.189369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063825d-2434-41b7-aa69-944bad92ab00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad64f2f20d3543429bec366a6034022c, user_id=e70c356a5992411189e31ced50c5868c, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:32.192037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063825d-2434-41b7-aa69-944bad92ab00 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:32.203961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 909/1860] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:32 2022] PUT /identity/v3/projects/ad64f2f20d3543429bec366a6034022c/users/e70c356a5992411189e31ced50c5868c/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:32.214189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10165f83-6a6b-410a-8544-385ad2e0f4a7 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:32.214469 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10165f83-6a6b-410a-8544-385ad2e0f4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:32.215321 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10165f83-6a6b-410a-8544-385ad2e0f4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:32.244175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-10165f83-6a6b-410a-8544-385ad2e0f4a7 None None] MFA Rules not processed for user `e70c356a5992411189e31ced50c5868c`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:32.245350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-10165f83-6a6b-410a-8544-385ad2e0f4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:32.319870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 952/1861] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:32.327254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063825d-2434-41b7-aa69-944bad92ab00 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:32.330001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2063825d-2434-41b7-aa69-944bad92ab00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:32.405010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063825d-2434-41b7-aa69-944bad92ab00 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:32.410713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb9d34-7ac7-4b8d-950d-7d7d2423853e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:32.411676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb9d34-7ac7-4b8d-950d-7d7d2423853e service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:32.412151 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb9d34-7ac7-4b8d-950d-7d7d2423853e service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:32.413237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb9d34-7ac7-4b8d-950d-7d7d2423853e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:32.424568 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb9d34-7ac7-4b8d-950d-7d7d2423853e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:32.426256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb9d34-7ac7-4b8d-950d-7d7d2423853e service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:32.436722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 910/1862] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:56:32.490928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10165f83-6a6b-410a-8544-385ad2e0f4a7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:32.492456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-10165f83-6a6b-410a-8544-385ad2e0f4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:32.505431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10165f83-6a6b-410a-8544-385ad2e0f4a7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:32.509462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330dc44-9305-479f-b4ca-2c66ce37e726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:32.514048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330dc44-9305-479f-b4ca-2c66ce37e726 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:32.514491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330dc44-9305-479f-b4ca-2c66ce37e726 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:32.514765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330dc44-9305-479f-b4ca-2c66ce37e726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:32.571484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330dc44-9305-479f-b4ca-2c66ce37e726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:32.573186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330dc44-9305-479f-b4ca-2c66ce37e726 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:32.582662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 953/1863] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:35.921962 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb9d34-7ac7-4b8d-950d-7d7d2423853e service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:35.926290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:35.993790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb9d34-7ac7-4b8d-950d-7d7d2423853e service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:35.998963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:35.999618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:35.999902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:36.000178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] PATH_INFO: `/v3/users/5395aaca2a81470ba45f7075c256ec68` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:36.001318 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5395aaca2a81470ba45f7075c256ec68)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:36.003210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:36.069293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5395aaca2a81470ba45f7075c256ec68'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.070078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5395aaca2a81470ba45f7075c256ec68'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.079904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5395aaca2a81470ba45f7075c256ec68'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.083176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5395aaca2a81470ba45f7075c256ec68'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.093385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5395aaca2a81470ba45f7075c256ec68'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.110529 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 911/1864] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:35 2022] DELETE /identity/v3/users/5395aaca2a81470ba45f7075c256ec68 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:36.115363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2410b-0fcd-4d89-aa56-b97e17a3f8a4 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:36.116132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2410b-0fcd-4d89-aa56-b97e17a3f8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:36.116132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2410b-0fcd-4d89-aa56-b97e17a3f8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:36.137477 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e5f2410b-0fcd-4d89-aa56-b97e17a3f8a4 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:36.138869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e5f2410b-0fcd-4d89-aa56-b97e17a3f8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:36.173779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:36.199508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 954/1865] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:36.209285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:36.210687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:36.267537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099c4fa-1e07-4bde-bdae-e8bdaec24a9f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:36.271092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db997b-23f2-4488-a484-1c069b88c6e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:36.271682 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db997b-23f2-4488-a484-1c069b88c6e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:36.271987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db997b-23f2-4488-a484-1c069b88c6e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:36.272253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db997b-23f2-4488-a484-1c069b88c6e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:36.281504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db997b-23f2-4488-a484-1c069b88c6e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:36.282902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db997b-23f2-4488-a484-1c069b88c6e7 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:36.292971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 912/1866] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:36.582906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2410b-0fcd-4d89-aa56-b97e17a3f8a4 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:36.584633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e5f2410b-0fcd-4d89-aa56-b97e17a3f8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:36.596901 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2410b-0fcd-4d89-aa56-b97e17a3f8a4 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:36.601872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:36.602529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:36.602805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:36.603069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] PATH_INFO: `/v3/projects/93ebc5636b0840c5960a9fd265c8f55a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:36.604260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ebc5636b0840c5960a9fd265c8f55a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:36.606448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:36.666251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ebc5636b0840c5960a9fd265c8f55a', 'request_id': 'req-cb182330-7f0e-48d5-a6a7-6e2721c924b3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.673387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ebc5636b0840c5960a9fd265c8f55a', 'request_id': 'req-cb182330-7f0e-48d5-a6a7-6e2721c924b3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.681728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ebc5636b0840c5960a9fd265c8f55a', 'request_id': 'req-cb182330-7f0e-48d5-a6a7-6e2721c924b3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.695934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] The token cache is being invalidate because project 93ebc5636b0840c5960a9fd265c8f55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:36.696222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.696874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.699961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 955/1867] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:36 2022] DELETE /identity/v3/projects/93ebc5636b0840c5960a9fd265c8f55a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:36.704002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db997b-23f2-4488-a484-1c069b88c6e7 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:36.713657 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db997b-23f2-4488-a484-1c069b88c6e7 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:36.735456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:36.783425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:36.784115 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:36.784393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:36.784916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] PATH_INFO: `/v3/users/e70c356a5992411189e31ced50c5868c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:36.786039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e70c356a5992411189e31ced50c5868c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:36.787974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:36.839468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e70c356a5992411189e31ced50c5868c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.840148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e70c356a5992411189e31ced50c5868c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.848467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e70c356a5992411189e31ced50c5868c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.849142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e70c356a5992411189e31ced50c5868c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.863538 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-668d88d8-5964-4065-8cd9-044bb2293a4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e70c356a5992411189e31ced50c5868c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:36.880802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 913/1868] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:36 2022] DELETE /identity/v3/users/e70c356a5992411189e31ced50c5868c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:37.203439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:37.205009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:37.239764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb182330-7f0e-48d5-a6a7-6e2721c924b3 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:37.308347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:37.308909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:37.309181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:37.309447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] PATH_INFO: `/v3/projects/ad64f2f20d3543429bec366a6034022c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:37.310544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad64f2f20d3543429bec366a6034022c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:37.312498 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:37.360654 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad64f2f20d3543429bec366a6034022c', 'request_id': 'req-14f824be-e33d-406e-a59f-3644c633596f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:37.366107 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad64f2f20d3543429bec366a6034022c', 'request_id': 'req-14f824be-e33d-406e-a59f-3644c633596f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:37.374055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad64f2f20d3543429bec366a6034022c', 'request_id': 'req-14f824be-e33d-406e-a59f-3644c633596f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:37.375432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:37.387626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] The token cache is being invalidate because project ad64f2f20d3543429bec366a6034022c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:37.388011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:37.389055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-14f824be-e33d-406e-a59f-3644c633596f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:37.390868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 956/1869] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:37 2022] DELETE /identity/v3/projects/ad64f2f20d3543429bec366a6034022c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:37.396977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fb226-3a7a-473b-8902-3300a5a51456 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:37.397452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fb226-3a7a-473b-8902-3300a5a51456 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:37.397649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fb226-3a7a-473b-8902-3300a5a51456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:37.418490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-510fb226-3a7a-473b-8902-3300a5a51456 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:37.419685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-510fb226-3a7a-473b-8902-3300a5a51456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:37.478247 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 914/1870] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:37.486850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0dbfe-dab9-4a8b-a2ee-3aad86168417 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:37.487270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0dbfe-dab9-4a8b-a2ee-3aad86168417 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:37.487637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0dbfe-dab9-4a8b-a2ee-3aad86168417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:37.508154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-31d0dbfe-dab9-4a8b-a2ee-3aad86168417 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:37.509352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-31d0dbfe-dab9-4a8b-a2ee-3aad86168417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:37.517846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 957/1871] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:37.522100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fb226-3a7a-473b-8902-3300a5a51456 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:37.532392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fb226-3a7a-473b-8902-3300a5a51456 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:37.537065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e4609-6580-4773-b912-0140c7550504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:37.537661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e4609-6580-4773-b912-0140c7550504 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:37.537934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e4609-6580-4773-b912-0140c7550504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:37.538199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e4609-6580-4773-b912-0140c7550504 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:37.538828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e4609-6580-4773-b912-0140c7550504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:37.539123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e4609-6580-4773-b912-0140c7550504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:37.540727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e4609-6580-4773-b912-0140c7550504 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:37.552885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 915/1872] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:56:37.556999 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0dbfe-dab9-4a8b-a2ee-3aad86168417 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:37.566844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0dbfe-dab9-4a8b-a2ee-3aad86168417 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:37.571444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a998-f500-4233-8ed7-305db5210e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:37.572078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a998-f500-4233-8ed7-305db5210e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:37.572354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a998-f500-4233-8ed7-305db5210e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:37.572623 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a998-f500-4233-8ed7-305db5210e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:37.573242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057a998-f500-4233-8ed7-305db5210e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:37.575226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057a998-f500-4233-8ed7-305db5210e16 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:37.602178 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 958/1873] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:37 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:37.606896 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e4609-6580-4773-b912-0140c7550504 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:37.616821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e4609-6580-4773-b912-0140c7550504 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:37.646108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:37.677165 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb096c44-6a1d-4195-ac4c-2015c4048cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:37.677774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb096c44-6a1d-4195-ac4c-2015c4048cef admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:37.678045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb096c44-6a1d-4195-ac4c-2015c4048cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:37.678310 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb096c44-6a1d-4195-ac4c-2015c4048cef admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:37.678956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb096c44-6a1d-4195-ac4c-2015c4048cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:37.680918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb096c44-6a1d-4195-ac4c-2015c4048cef admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:37.703143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 916/1874] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:37 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:37.707098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a998-f500-4233-8ed7-305db5210e16 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:37.718276 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a998-f500-4233-8ed7-305db5210e16 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:37.723144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad7cef-74ca-4861-9916-316519d22680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:37.723920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad7cef-74ca-4861-9916-316519d22680 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:37.724232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad7cef-74ca-4861-9916-316519d22680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:37.724535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad7cef-74ca-4861-9916-316519d22680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:37.725175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad7cef-74ca-4861-9916-316519d22680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:37.727905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad7cef-74ca-4861-9916-316519d22680 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:37.739431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 959/1875] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:37.743489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb096c44-6a1d-4195-ac4c-2015c4048cef admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:37.754906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb096c44-6a1d-4195-ac4c-2015c4048cef admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:37.759859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f35fd8-26ca-40e0-8798-29b3186a0371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:37.761308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f35fd8-26ca-40e0-8798-29b3186a0371 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:37.761308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f35fd8-26ca-40e0-8798-29b3186a0371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:37.761503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f35fd8-26ca-40e0-8798-29b3186a0371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:37.762108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f35fd8-26ca-40e0-8798-29b3186a0371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:37.764016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f35fd8-26ca-40e0-8798-29b3186a0371 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:37.775726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 917/1876] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:37.780600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad7cef-74ca-4861-9916-316519d22680 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:37.791294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad7cef-74ca-4861-9916-316519d22680 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:37.796215 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a97e5e-323a-4d90-a8a4-1f1f8db77030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:37.796860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a97e5e-323a-4d90-a8a4-1f1f8db77030 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:37.797135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a97e5e-323a-4d90-a8a4-1f1f8db77030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:37.797405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a97e5e-323a-4d90-a8a4-1f1f8db77030 admin admin] PATH_INFO: `/v3/projects/4d7c32955b2349818a48d6dcacf1ec2b/users/492440f9091e4f8a86881b37c54a0cc6/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:37.812818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a97e5e-323a-4d90-a8a4-1f1f8db77030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d7c32955b2349818a48d6dcacf1ec2b, user_id=492440f9091e4f8a86881b37c54a0cc6, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:37.815481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a97e5e-323a-4d90-a8a4-1f1f8db77030 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:37.834093 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 960/1877] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:37 2022] PUT /identity/v3/projects/4d7c32955b2349818a48d6dcacf1ec2b/users/492440f9091e4f8a86881b37c54a0cc6/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:37.845999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060d7f9-5082-4594-a51d-102c032ddfe7 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:37.846372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060d7f9-5082-4594-a51d-102c032ddfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:37.846748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060d7f9-5082-4594-a51d-102c032ddfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:37.874337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-0060d7f9-5082-4594-a51d-102c032ddfe7 None None] MFA Rules not processed for user `492440f9091e4f8a86881b37c54a0cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:37.875513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0060d7f9-5082-4594-a51d-102c032ddfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:37.938958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 918/1878] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:37 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:37.943561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a97e5e-323a-4d90-a8a4-1f1f8db77030 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.008529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a97e5e-323a-4d90-a8a4-1f1f8db77030 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:38.014313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259e933-e9de-471b-87ad-c3417c1d457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:38.014923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259e933-e9de-471b-87ad-c3417c1d457e admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.015194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259e933-e9de-471b-87ad-c3417c1d457e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.015456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259e933-e9de-471b-87ad-c3417c1d457e admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.016121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259e933-e9de-471b-87ad-c3417c1d457e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:38.018082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259e933-e9de-471b-87ad-c3417c1d457e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:38.052986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 961/1879] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:37 2022] POST /identity/v3/projects => generated 399 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:38.056808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060d7f9-5082-4594-a51d-102c032ddfe7 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.076948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060d7f9-5082-4594-a51d-102c032ddfe7 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:38.140176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d279ce-7134-474f-be1a-53be0dd05987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:38.140789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d279ce-7134-474f-be1a-53be0dd05987 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.141068 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d279ce-7134-474f-be1a-53be0dd05987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.141340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d279ce-7134-474f-be1a-53be0dd05987 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.141969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d279ce-7134-474f-be1a-53be0dd05987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:38.143904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d279ce-7134-474f-be1a-53be0dd05987 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:38.165195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 919/1880] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:38 2022] POST /identity/v3/users => generated 306 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:38.169483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259e933-e9de-471b-87ad-c3417c1d457e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.179569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259e933-e9de-471b-87ad-c3417c1d457e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:38.185080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb049de-096c-45f8-81fa-ba70fae829ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:38.185684 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb049de-096c-45f8-81fa-ba70fae829ee admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.185962 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb049de-096c-45f8-81fa-ba70fae829ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.186234 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb049de-096c-45f8-81fa-ba70fae829ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.186913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb049de-096c-45f8-81fa-ba70fae829ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:38.188573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb049de-096c-45f8-81fa-ba70fae829ee admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:38.200495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 962/1881] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:38.204636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d279ce-7134-474f-be1a-53be0dd05987 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.214926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d279ce-7134-474f-be1a-53be0dd05987 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:38.219465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2209f99-1d88-4879-91a8-06f0a5754e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:38.220141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2209f99-1d88-4879-91a8-06f0a5754e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.220423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2209f99-1d88-4879-91a8-06f0a5754e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.220691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2209f99-1d88-4879-91a8-06f0a5754e51 admin admin] PATH_INFO: `/v3/projects/a7ba051cc5f443d8b9a40316d6d611d7/users/9ce792cab996437eaf0abca4a89aac8f/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.235661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2209f99-1d88-4879-91a8-06f0a5754e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7ba051cc5f443d8b9a40316d6d611d7, user_id=9ce792cab996437eaf0abca4a89aac8f, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:38.238605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2209f99-1d88-4879-91a8-06f0a5754e51 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:38.252098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 920/1882] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:38 2022] PUT /identity/v3/projects/a7ba051cc5f443d8b9a40316d6d611d7/users/9ce792cab996437eaf0abca4a89aac8f/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:38.255999 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb049de-096c-45f8-81fa-ba70fae829ee admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.267691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb049de-096c-45f8-81fa-ba70fae829ee admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:38.328626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baea381-0c80-411c-86db-30a0f65ed10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:38.332104 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baea381-0c80-411c-86db-30a0f65ed10c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.332387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baea381-0c80-411c-86db-30a0f65ed10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.332666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baea381-0c80-411c-86db-30a0f65ed10c admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.332929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2209f99-1d88-4879-91a8-06f0a5754e51 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.333318 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baea381-0c80-411c-86db-30a0f65ed10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:38.335230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f2209f99-1d88-4879-91a8-06f0a5754e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:38.335380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baea381-0c80-411c-86db-30a0f65ed10c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:38.341135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:38.345448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:38.348932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 963/1883] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:38.351136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2209f99-1d88-4879-91a8-06f0a5754e51 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:38.353299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baea381-0c80-411c-86db-30a0f65ed10c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.360387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:38.361128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.361718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.362323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] PATH_INFO: `/v3/users/342c9e930d834316bcd2997476370d2e` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.364593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=342c9e930d834316bcd2997476370d2e)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:38.366544 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:38.367195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baea381-0c80-411c-86db-30a0f65ed10c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:38.373581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8696efe-9479-419e-8652-b1fe5716c8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:38.375066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8696efe-9479-419e-8652-b1fe5716c8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.375346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8696efe-9479-419e-8652-b1fe5716c8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.375621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8696efe-9479-419e-8652-b1fe5716c8b2 admin admin] PATH_INFO: `/v3/projects/a7ba051cc5f443d8b9a40316d6d611d7/users/9ce792cab996437eaf0abca4a89aac8f/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.377597 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8696efe-9479-419e-8652-b1fe5716c8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7ba051cc5f443d8b9a40316d6d611d7, user_id=9ce792cab996437eaf0abca4a89aac8f, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:38.380741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8696efe-9479-419e-8652-b1fe5716c8b2 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:38.394788 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 964/1884] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:38 2022] PUT /identity/v3/projects/a7ba051cc5f443d8b9a40316d6d611d7/users/9ce792cab996437eaf0abca4a89aac8f/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:38.404859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511bcf-1bed-47c4-a922-b3c7b57d3d09 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.405133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511bcf-1bed-47c4-a922-b3c7b57d3d09 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.405432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511bcf-1bed-47c4-a922-b3c7b57d3d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.429702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342c9e930d834316bcd2997476370d2e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:38.430653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342c9e930d834316bcd2997476370d2e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:38.436294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-14511bcf-1bed-47c4-a922-b3c7b57d3d09 None None] MFA Rules not processed for user `9ce792cab996437eaf0abca4a89aac8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:38.437517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-14511bcf-1bed-47c4-a922-b3c7b57d3d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:38.441045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342c9e930d834316bcd2997476370d2e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:38.441895 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342c9e930d834316bcd2997476370d2e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:38.457644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c7e81a0b-905a-43fd-bc6b-19e429257cd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342c9e930d834316bcd2997476370d2e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:38.477621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 921/1885] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:38 2022] DELETE /identity/v3/users/342c9e930d834316bcd2997476370d2e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:38.480524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f61c25-fc51-461e-bd64-831fa18422ce None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.480644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f61c25-fc51-461e-bd64-831fa18422ce None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.480834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f61c25-fc51-461e-bd64-831fa18422ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.503086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-86f61c25-fc51-461e-bd64-831fa18422ce None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:38.504344 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-86f61c25-fc51-461e-bd64-831fa18422ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:38.522836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 965/1886] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:38 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:38.534453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14511bcf-1bed-47c4-a922-b3c7b57d3d09 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.536085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-14511bcf-1bed-47c4-a922-b3c7b57d3d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:38.564581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 922/1887] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:38.575006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f61c25-fc51-461e-bd64-831fa18422ce None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.576369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-86f61c25-fc51-461e-bd64-831fa18422ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:38.596021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14511bcf-1bed-47c4-a922-b3c7b57d3d09 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:38.633060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f61c25-fc51-461e-bd64-831fa18422ce None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:38.637209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12954a51-65f5-4fd8-8387-3fc6a0f10cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:38.637820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12954a51-65f5-4fd8-8387-3fc6a0f10cbb service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.638098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12954a51-65f5-4fd8-8387-3fc6a0f10cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.638367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12954a51-65f5-4fd8-8387-3fc6a0f10cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.643387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891261a-a85b-454f-acaf-2b228d3ab315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:38.644095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891261a-a85b-454f-acaf-2b228d3ab315 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.644320 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891261a-a85b-454f-acaf-2b228d3ab315 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.644592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891261a-a85b-454f-acaf-2b228d3ab315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.647385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12954a51-65f5-4fd8-8387-3fc6a0f10cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:38.648959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12954a51-65f5-4fd8-8387-3fc6a0f10cbb service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:38.659139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 923/1888] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:56:38.686802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:38.709594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891261a-a85b-454f-acaf-2b228d3ab315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:38.711187 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891261a-a85b-454f-acaf-2b228d3ab315 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:38.721793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 966/1889] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:38 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 17:56:38.801357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12954a51-65f5-4fd8-8387-3fc6a0f10cbb service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.811017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12954a51-65f5-4fd8-8387-3fc6a0f10cbb service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:38.814830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252dd6d-27cd-47e1-a9ff-c48a8d22a1ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:38.815423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252dd6d-27cd-47e1-a9ff-c48a8d22a1ae service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:38.815697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252dd6d-27cd-47e1-a9ff-c48a8d22a1ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:38.816001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252dd6d-27cd-47e1-a9ff-c48a8d22a1ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:38.818158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a252dd6d-27cd-47e1-a9ff-c48a8d22a1ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:38.838428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:38.891410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252dd6d-27cd-47e1-a9ff-c48a8d22a1ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:38.892935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252dd6d-27cd-47e1-a9ff-c48a8d22a1ae service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:38.903972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 924/1890] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:38 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:38.967307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891261a-a85b-454f-acaf-2b228d3ab315 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:38.968751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b891261a-a85b-454f-acaf-2b228d3ab315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:38.993061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891261a-a85b-454f-acaf-2b228d3ab315 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:39.000599 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:39.001251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.002186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.002420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] PATH_INFO: `/v3/projects/fd58299057494f4d9bf89b7068943f8b` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.003629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd58299057494f4d9bf89b7068943f8b)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:39.005766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:39.062917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd58299057494f4d9bf89b7068943f8b', 'request_id': 'req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:39.069278 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd58299057494f4d9bf89b7068943f8b', 'request_id': 'req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:39.078509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd58299057494f4d9bf89b7068943f8b', 'request_id': 'req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:39.092992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] The token cache is being invalidate because project fd58299057494f4d9bf89b7068943f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:39.093427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:39.094330 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-9decb5c2-5be8-4f39-9d90-3dd9676c57cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:39.096158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 967/1891] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:38 2022] DELETE /identity/v3/projects/fd58299057494f4d9bf89b7068943f8b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:39.101447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe50bb-f695-416a-911d-9a66141a788c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.101787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe50bb-f695-416a-911d-9a66141a788c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.102062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe50bb-f695-416a-911d-9a66141a788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.124860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-62fe50bb-f695-416a-911d-9a66141a788c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:39.126063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-62fe50bb-f695-416a-911d-9a66141a788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:39.192933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 925/1892] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:39.210319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9bb50-5c0d-40cd-bc84-ef5aa3514be9 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.210319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9bb50-5c0d-40cd-bc84-ef5aa3514be9 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.210319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9bb50-5c0d-40cd-bc84-ef5aa3514be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.237706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-9df9bb50-5c0d-40cd-bc84-ef5aa3514be9 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:39.238900 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9df9bb50-5c0d-40cd-bc84-ef5aa3514be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:39.247211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 968/1893] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:39.252468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe50bb-f695-416a-911d-9a66141a788c None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:39.263987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe50bb-f695-416a-911d-9a66141a788c None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:39.271974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18237cff-3951-42fa-85b9-6c5ec82ef957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:39.272670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18237cff-3951-42fa-85b9-6c5ec82ef957 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.273034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18237cff-3951-42fa-85b9-6c5ec82ef957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.273365 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18237cff-3951-42fa-85b9-6c5ec82ef957 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.274077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18237cff-3951-42fa-85b9-6c5ec82ef957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:39.274441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18237cff-3951-42fa-85b9-6c5ec82ef957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:39.276144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18237cff-3951-42fa-85b9-6c5ec82ef957 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:39.289924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 926/1894] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:56:39.294170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9bb50-5c0d-40cd-bc84-ef5aa3514be9 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:39.304987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9bb50-5c0d-40cd-bc84-ef5aa3514be9 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:39.309888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5385e-1d3a-431c-9e98-8295b3438937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:39.310481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5385e-1d3a-431c-9e98-8295b3438937 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.310752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5385e-1d3a-431c-9e98-8295b3438937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.311015 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5385e-1d3a-431c-9e98-8295b3438937 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.311692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5385e-1d3a-431c-9e98-8295b3438937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:39.313653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5385e-1d3a-431c-9e98-8295b3438937 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:39.344771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 969/1895] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:39 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:39.345888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18237cff-3951-42fa-85b9-6c5ec82ef957 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:39.356381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18237cff-3951-42fa-85b9-6c5ec82ef957 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:39.413232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0523bc-b5b8-4531-97c1-dbf4d0a46dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:39.413760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0523bc-b5b8-4531-97c1-dbf4d0a46dbd admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.414036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0523bc-b5b8-4531-97c1-dbf4d0a46dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.414312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0523bc-b5b8-4531-97c1-dbf4d0a46dbd admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.414945 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0523bc-b5b8-4531-97c1-dbf4d0a46dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:39.416897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0523bc-b5b8-4531-97c1-dbf4d0a46dbd admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:39.440245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 927/1896] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:39 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:39.441581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5385e-1d3a-431c-9e98-8295b3438937 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:39.452535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5385e-1d3a-431c-9e98-8295b3438937 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:39.457007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c00ad-f7ce-448d-afd4-b0239ee2bbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:39.457588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c00ad-f7ce-448d-afd4-b0239ee2bbbe admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.457854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c00ad-f7ce-448d-afd4-b0239ee2bbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.458113 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c00ad-f7ce-448d-afd4-b0239ee2bbbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.458767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c00ad-f7ce-448d-afd4-b0239ee2bbbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:39.460387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c00ad-f7ce-448d-afd4-b0239ee2bbbe admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:39.470947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 970/1897] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:39.474772 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0523bc-b5b8-4531-97c1-dbf4d0a46dbd admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:39.485518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0523bc-b5b8-4531-97c1-dbf4d0a46dbd admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:39.489971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bacb4-9a93-4f09-a987-facf687ed754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:39.490553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bacb4-9a93-4f09-a987-facf687ed754 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.490817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bacb4-9a93-4f09-a987-facf687ed754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.491087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bacb4-9a93-4f09-a987-facf687ed754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.491713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bacb4-9a93-4f09-a987-facf687ed754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:39.493320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bacb4-9a93-4f09-a987-facf687ed754 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:39.504264 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 928/1898] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:39.508366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c00ad-f7ce-448d-afd4-b0239ee2bbbe admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:39.520056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c00ad-f7ce-448d-afd4-b0239ee2bbbe admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:39.525376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53d20e-2840-41e3-86ef-4d7bac40ed0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:39.526113 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53d20e-2840-41e3-86ef-4d7bac40ed0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.526454 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53d20e-2840-41e3-86ef-4d7bac40ed0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.527274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53d20e-2840-41e3-86ef-4d7bac40ed0b admin admin] PATH_INFO: `/v3/projects/c66687b1c51d42ca9cc0025c29cf1e3a/users/8f4de098424a40f4a384786966abdd7d/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.545387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53d20e-2840-41e3-86ef-4d7bac40ed0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66687b1c51d42ca9cc0025c29cf1e3a, user_id=8f4de098424a40f4a384786966abdd7d, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:39.548657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53d20e-2840-41e3-86ef-4d7bac40ed0b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:39.578103 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 971/1899] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:39 2022] PUT /identity/v3/projects/c66687b1c51d42ca9cc0025c29cf1e3a/users/8f4de098424a40f4a384786966abdd7d/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:39.588645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6f421-65ff-44b7-a79a-4bfa6698790a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.590252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6f421-65ff-44b7-a79a-4bfa6698790a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.590609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6f421-65ff-44b7-a79a-4bfa6698790a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.622396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ebe6f421-65ff-44b7-a79a-4bfa6698790a None None] MFA Rules not processed for user `8f4de098424a40f4a384786966abdd7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:39.623620 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ebe6f421-65ff-44b7-a79a-4bfa6698790a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:39.635606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:39.693138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 929/1900] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:39.705340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53d20e-2840-41e3-86ef-4d7bac40ed0b admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:39.707011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5d53d20e-2840-41e3-86ef-4d7bac40ed0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:39.764356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53d20e-2840-41e3-86ef-4d7bac40ed0b admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:39.767927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac20fd6-18ba-42ef-b3be-a7fe57749c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:39.768518 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac20fd6-18ba-42ef-b3be-a7fe57749c0f service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:39.768788 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac20fd6-18ba-42ef-b3be-a7fe57749c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:39.769050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac20fd6-18ba-42ef-b3be-a7fe57749c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:39.777531 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac20fd6-18ba-42ef-b3be-a7fe57749c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:39.778941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac20fd6-18ba-42ef-b3be-a7fe57749c0f service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:39.783271 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:39.792018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 972/1901] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:43.180379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6f421-65ff-44b7-a79a-4bfa6698790a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:43.181835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ebe6f421-65ff-44b7-a79a-4bfa6698790a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:43.262806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6f421-65ff-44b7-a79a-4bfa6698790a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:43.267465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:43.268242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:43.268522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:43.268796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] PATH_INFO: `/v3/users/492440f9091e4f8a86881b37c54a0cc6` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:43.269928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] RBAC: Authorizing `identity:delete_user(user_id=492440f9091e4f8a86881b37c54a0cc6)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:43.271847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:43.321495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '492440f9091e4f8a86881b37c54a0cc6'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:43.322181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '492440f9091e4f8a86881b37c54a0cc6'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:43.331029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '492440f9091e4f8a86881b37c54a0cc6'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:43.331754 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '492440f9091e4f8a86881b37c54a0cc6'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:43.350638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '492440f9091e4f8a86881b37c54a0cc6'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:43.374794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 930/1902] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:43 2022] DELETE /identity/v3/users/492440f9091e4f8a86881b37c54a0cc6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:43.379212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c93d0-c123-4d34-8683-3db66f3a3dd8 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:43.379733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c93d0-c123-4d34-8683-3db66f3a3dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:43.380093 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c93d0-c123-4d34-8683-3db66f3a3dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:43.400108 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-6e5c93d0-c123-4d34-8683-3db66f3a3dd8 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:43.401331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6e5c93d0-c123-4d34-8683-3db66f3a3dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:43.468588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 973/1903] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:43.480824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:43.482490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:43.530549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:43.559154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca6cdb-45ae-47e0-b30b-e206fdb9a30a admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:43.564476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adaf63-c68d-47fe-8097-b6b19472c4d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:43.565624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adaf63-c68d-47fe-8097-b6b19472c4d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:43.565624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adaf63-c68d-47fe-8097-b6b19472c4d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:43.565833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adaf63-c68d-47fe-8097-b6b19472c4d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:43.578390 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adaf63-c68d-47fe-8097-b6b19472c4d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:43.580468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adaf63-c68d-47fe-8097-b6b19472c4d6 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:43.594350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 931/1904] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:43.919381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c93d0-c123-4d34-8683-3db66f3a3dd8 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:43.920887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6e5c93d0-c123-4d34-8683-3db66f3a3dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:43.932462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c93d0-c123-4d34-8683-3db66f3a3dd8 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:43.937782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:43.938410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:43.938681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:43.938954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] PATH_INFO: `/v3/projects/4d7c32955b2349818a48d6dcacf1ec2b` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:43.940084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d7c32955b2349818a48d6dcacf1ec2b)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:43.942286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:43.999834 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d7c32955b2349818a48d6dcacf1ec2b', 'request_id': 'req-04c332aa-71d1-4782-b171-caae4efb71f6'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.005587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d7c32955b2349818a48d6dcacf1ec2b', 'request_id': 'req-04c332aa-71d1-4782-b171-caae4efb71f6'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.013314 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d7c32955b2349818a48d6dcacf1ec2b', 'request_id': 'req-04c332aa-71d1-4782-b171-caae4efb71f6'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.024555 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] The token cache is being invalidate because project 4d7c32955b2349818a48d6dcacf1ec2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:44.024838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.025988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.028003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 974/1905] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:43 2022] DELETE /identity/v3/projects/4d7c32955b2349818a48d6dcacf1ec2b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:44.035725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adaf63-c68d-47fe-8097-b6b19472c4d6 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:44.050575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adaf63-c68d-47fe-8097-b6b19472c4d6 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:44.111557 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:44.112521 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:44.113041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:44.113365 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] PATH_INFO: `/v3/users/9ce792cab996437eaf0abca4a89aac8f` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:44.114781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ce792cab996437eaf0abca4a89aac8f)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:44.117094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:44.182095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ce792cab996437eaf0abca4a89aac8f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.183334 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ce792cab996437eaf0abca4a89aac8f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.190888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ce792cab996437eaf0abca4a89aac8f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.191524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ce792cab996437eaf0abca4a89aac8f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.204186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4a1306d3-d5b2-4697-a714-d7d8609d62ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ce792cab996437eaf0abca4a89aac8f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.218534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 932/1906] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:44 2022] DELETE /identity/v3/users/9ce792cab996437eaf0abca4a89aac8f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:44.555623 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:44.557412 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:44.581790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c332aa-71d1-4782-b171-caae4efb71f6 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:44.646703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:44.647477 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:44.647821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:44.648190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] PATH_INFO: `/v3/projects/a7ba051cc5f443d8b9a40316d6d611d7` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:44.649373 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7ba051cc5f443d8b9a40316d6d611d7)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:44.651345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:44.686233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:44.708100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7ba051cc5f443d8b9a40316d6d611d7', 'request_id': 'req-7f15f4ef-2e54-4a37-8f96-171143161f55'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.710513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7ba051cc5f443d8b9a40316d6d611d7', 'request_id': 'req-7f15f4ef-2e54-4a37-8f96-171143161f55'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.719079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7ba051cc5f443d8b9a40316d6d611d7', 'request_id': 'req-7f15f4ef-2e54-4a37-8f96-171143161f55'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.731598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] The token cache is being invalidate because project a7ba051cc5f443d8b9a40316d6d611d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:44.731906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.732564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7f15f4ef-2e54-4a37-8f96-171143161f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:44.734309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 975/1907] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:44 2022] DELETE /identity/v3/projects/a7ba051cc5f443d8b9a40316d6d611d7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:44.739246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a17d71-e265-4672-befd-3b6152f94a5d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:44.739598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a17d71-e265-4672-befd-3b6152f94a5d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:44.739979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a17d71-e265-4672-befd-3b6152f94a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:44.759906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-42a17d71-e265-4672-befd-3b6152f94a5d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:44.762111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-42a17d71-e265-4672-befd-3b6152f94a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:44.811155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:44.825652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 933/1908] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:44.834715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c605c-9fde-4f94-a30b-940cf49444c7 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:44.835658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c605c-9fde-4f94-a30b-940cf49444c7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:44.836259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c605c-9fde-4f94-a30b-940cf49444c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:44.858346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-491c605c-9fde-4f94-a30b-940cf49444c7 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:44.859537 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-491c605c-9fde-4f94-a30b-940cf49444c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:44.867308 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 976/1909] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:44.873646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a17d71-e265-4672-befd-3b6152f94a5d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:44.883258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a17d71-e265-4672-befd-3b6152f94a5d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:44.888813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b6abb-9e49-4714-b950-c074317d9c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:44.889408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b6abb-9e49-4714-b950-c074317d9c1e admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:44.889683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b6abb-9e49-4714-b950-c074317d9c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:44.889950 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b6abb-9e49-4714-b950-c074317d9c1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:44.890591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b6abb-9e49-4714-b950-c074317d9c1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:44.890885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b6abb-9e49-4714-b950-c074317d9c1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:44.892497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b6abb-9e49-4714-b950-c074317d9c1e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:44.907936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 934/1910] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:56:44.908869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c605c-9fde-4f94-a30b-940cf49444c7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:44.918403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c605c-9fde-4f94-a30b-940cf49444c7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:44.923072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316adb42-5f5c-47ea-881b-1f96f5676ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:44.923662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316adb42-5f5c-47ea-881b-1f96f5676ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:44.923973 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316adb42-5f5c-47ea-881b-1f96f5676ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:44.924240 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316adb42-5f5c-47ea-881b-1f96f5676ad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:44.924860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316adb42-5f5c-47ea-881b-1f96f5676ad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:44.926746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316adb42-5f5c-47ea-881b-1f96f5676ad8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:44.958836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 977/1911] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:44 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:44.960484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b6abb-9e49-4714-b950-c074317d9c1e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:44.970443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b6abb-9e49-4714-b950-c074317d9c1e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:45.028097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882e71b-1512-4ac7-b8a6-496190646310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:45.028708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882e71b-1512-4ac7-b8a6-496190646310 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:45.028985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882e71b-1512-4ac7-b8a6-496190646310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:45.029374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882e71b-1512-4ac7-b8a6-496190646310 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:45.030027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882e71b-1512-4ac7-b8a6-496190646310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:45.031994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882e71b-1512-4ac7-b8a6-496190646310 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:45.054233 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 935/1912] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:44 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:45.058914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316adb42-5f5c-47ea-881b-1f96f5676ad8 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:45.069047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316adb42-5f5c-47ea-881b-1f96f5676ad8 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:45.074816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230f93f-0da4-45b9-a9c9-38a82cf38b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:45.075413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230f93f-0da4-45b9-a9c9-38a82cf38b76 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:45.075684 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230f93f-0da4-45b9-a9c9-38a82cf38b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:45.076049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230f93f-0da4-45b9-a9c9-38a82cf38b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:45.076685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230f93f-0da4-45b9-a9c9-38a82cf38b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:45.078264 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230f93f-0da4-45b9-a9c9-38a82cf38b76 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:45.091555 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 978/1913] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:45.092910 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882e71b-1512-4ac7-b8a6-496190646310 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:45.102922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882e71b-1512-4ac7-b8a6-496190646310 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:45.108739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76635164-73c0-41fc-9331-043577577225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:45.109399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635164-73c0-41fc-9331-043577577225 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:45.109738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635164-73c0-41fc-9331-043577577225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:45.110091 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635164-73c0-41fc-9331-043577577225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:45.110785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76635164-73c0-41fc-9331-043577577225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:45.112536 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76635164-73c0-41fc-9331-043577577225 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:45.125221 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 936/1914] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:45.129634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230f93f-0da4-45b9-a9c9-38a82cf38b76 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:45.143868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230f93f-0da4-45b9-a9c9-38a82cf38b76 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:45.150788 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d7fcf-90d3-402f-9b70-589f09db5736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:45.151637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d7fcf-90d3-402f-9b70-589f09db5736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:45.152125 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d7fcf-90d3-402f-9b70-589f09db5736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:45.152462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d7fcf-90d3-402f-9b70-589f09db5736 admin admin] PATH_INFO: `/v3/projects/2508480db6a848fe8bc4dae99c55bb9c/users/d0acb86eeea9488ab2cf6d987e972010/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:45.170543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d7fcf-90d3-402f-9b70-589f09db5736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2508480db6a848fe8bc4dae99c55bb9c, user_id=d0acb86eeea9488ab2cf6d987e972010, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:45.173468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d7fcf-90d3-402f-9b70-589f09db5736 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:45.188031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 979/1915] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:45 2022] PUT /identity/v3/projects/2508480db6a848fe8bc4dae99c55bb9c/users/d0acb86eeea9488ab2cf6d987e972010/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:45.198182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43718360-5c43-4b79-8b4d-6e5eac3c3af2 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:45.199149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43718360-5c43-4b79-8b4d-6e5eac3c3af2 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:45.199571 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43718360-5c43-4b79-8b4d-6e5eac3c3af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:45.231296 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-43718360-5c43-4b79-8b4d-6e5eac3c3af2 None None] MFA Rules not processed for user `d0acb86eeea9488ab2cf6d987e972010`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:45.232574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-43718360-5c43-4b79-8b4d-6e5eac3c3af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:45.303046 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 937/1916] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:45.314940 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d7fcf-90d3-402f-9b70-589f09db5736 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:45.316910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6c1d7fcf-90d3-402f-9b70-589f09db5736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:45.385140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d7fcf-90d3-402f-9b70-589f09db5736 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:45.389240 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0eed21-c827-4264-ae0b-2923a5d413e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3566d85c9c4c559ba672269ab38f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:45.389861 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eed21-c827-4264-ae0b-2923a5d413e1 service glance] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:45.390201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eed21-c827-4264-ae0b-2923a5d413e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:45.390536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eed21-c827-4264-ae0b-2923a5d413e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:45.399377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0eed21-c827-4264-ae0b-2923a5d413e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:45.400847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0eed21-c827-4264-ae0b-2923a5d413e1 service glance] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:45.415073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 980/1917] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:45.715222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0187e-784d-4685-af10-3eab96069b49 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:45.715717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0187e-784d-4685-af10-3eab96069b49 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:45.716316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0187e-784d-4685-af10-3eab96069b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:45.737730 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-0ee0187e-784d-4685-af10-3eab96069b49 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:45.739267 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0ee0187e-784d-4685-af10-3eab96069b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:45.790317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:45.804444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 938/1918] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:56:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:45.853116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0eed21-c827-4264-ae0b-2923a5d413e1 service glance] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:45.855387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-bf0eed21-c827-4264-ae0b-2923a5d413e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:45.926481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:45.950845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0eed21-c827-4264-ae0b-2923a5d413e1 service glance] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:45.956588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fc33e-2d47-4dd9-8537-356cd746b19c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:45.957217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fc33e-2d47-4dd9-8537-356cd746b19c service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:45.957469 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fc33e-2d47-4dd9-8537-356cd746b19c service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:45.957747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fc33e-2d47-4dd9-8537-356cd746b19c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:45.966857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fc33e-2d47-4dd9-8537-356cd746b19c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:45.968574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fc33e-2d47-4dd9-8537-356cd746b19c service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:45.977997 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 981/1919] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:56:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:56:46.626858 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d2b98-2008-4687-8961-1cac5d715d43 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:46.627330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d2b98-2008-4687-8961-1cac5d715d43 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:46.627330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d2b98-2008-4687-8961-1cac5d715d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:46.662574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-3a6d2b98-2008-4687-8961-1cac5d715d43 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:46.663678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3a6d2b98-2008-4687-8961-1cac5d715d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:46.679174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 939/1920] 10.222.0.27 () {58 vars in 967 bytes} [Fri Oct 14 17:56:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:46.696216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fc33e-2d47-4dd9-8537-356cd746b19c service swift] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:46.711252 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fc33e-2d47-4dd9-8537-356cd746b19c service swift] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:46.717651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a2305-6abe-4c9a-826a-b7b07f78ae54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:46.719033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a2305-6abe-4c9a-826a-b7b07f78ae54 service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:46.719384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a2305-6abe-4c9a-826a-b7b07f78ae54 service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:46.719727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a2305-6abe-4c9a-826a-b7b07f78ae54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:46.728886 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a2305-6abe-4c9a-826a-b7b07f78ae54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:46.730603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a2305-6abe-4c9a-826a-b7b07f78ae54 service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:46.739084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 982/1921] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:56:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:56:47.024585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560fe36-5eaf-4c49-a4b3-1ff7c2428fd9 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:47.024936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560fe36-5eaf-4c49-a4b3-1ff7c2428fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:47.025208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560fe36-5eaf-4c49-a4b3-1ff7c2428fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:47.046341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1560fe36-5eaf-4c49-a4b3-1ff7c2428fd9 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:47.047801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1560fe36-5eaf-4c49-a4b3-1ff7c2428fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:47.056251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 940/1922] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:56:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:47.069466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a2305-6abe-4c9a-826a-b7b07f78ae54 service swift] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:47.080715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a2305-6abe-4c9a-826a-b7b07f78ae54 service swift] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:47.085663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedba9d-5d3f-47ac-809e-3509b133a2a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:47.086249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedba9d-5d3f-47ac-809e-3509b133a2a7 service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:47.086520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedba9d-5d3f-47ac-809e-3509b133a2a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:47.086784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedba9d-5d3f-47ac-809e-3509b133a2a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:47.095157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedba9d-5d3f-47ac-809e-3509b133a2a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:47.096754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedba9d-5d3f-47ac-809e-3509b133a2a7 service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:47.105315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 983/1923] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:56:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:56:47.460396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ed957-24fc-4d47-bba1-039818e42097 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:47.460588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ed957-24fc-4d47-bba1-039818e42097 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:47.460854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ed957-24fc-4d47-bba1-039818e42097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:47.480774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-228ed957-24fc-4d47-bba1-039818e42097 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:47.481998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-228ed957-24fc-4d47-bba1-039818e42097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:47.489759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 941/1924] 10.222.0.27 () {58 vars in 967 bytes} [Fri Oct 14 17:56:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:47.502663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedba9d-5d3f-47ac-809e-3509b133a2a7 service swift] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:47.513247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedba9d-5d3f-47ac-809e-3509b133a2a7 service swift] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:47.518096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e85bc-6d84-4626-841e-5ff3292ade7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:47.518682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e85bc-6d84-4626-841e-5ff3292ade7a service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:47.518954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e85bc-6d84-4626-841e-5ff3292ade7a service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:47.519242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e85bc-6d84-4626-841e-5ff3292ade7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:47.527092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e85bc-6d84-4626-841e-5ff3292ade7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:47.528694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e85bc-6d84-4626-841e-5ff3292ade7a service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:47.536700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 984/1925] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:56:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:56:49.803780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ed957-24fc-4d47-bba1-039818e42097 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:49.805217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-228ed957-24fc-4d47-bba1-039818e42097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:49.873054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ed957-24fc-4d47-bba1-039818e42097 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:49.877539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:49.878151 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:49.878426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:49.878722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] PATH_INFO: `/v3/users/f676b3649fd04f699a5c5758fdc94017` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:49.879848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f676b3649fd04f699a5c5758fdc94017)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:49.881787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:49.930128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f676b3649fd04f699a5c5758fdc94017'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:49.930785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f676b3649fd04f699a5c5758fdc94017'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:49.938390 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f676b3649fd04f699a5c5758fdc94017'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:49.939024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f676b3649fd04f699a5c5758fdc94017'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:49.950981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f676b3649fd04f699a5c5758fdc94017'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:49.964048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 942/1926] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:49 2022] DELETE /identity/v3/users/f676b3649fd04f699a5c5758fdc94017 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:49.969279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd83a8f-1ea1-4f51-8cd0-7b0bcd803219 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:49.969754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd83a8f-1ea1-4f51-8cd0-7b0bcd803219 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:49.970002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd83a8f-1ea1-4f51-8cd0-7b0bcd803219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:49.992061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0cd83a8f-1ea1-4f51-8cd0-7b0bcd803219 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:49.993399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0cd83a8f-1ea1-4f51-8cd0-7b0bcd803219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:50.053219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 985/1927] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:50.063056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.064511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:50.124401 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12c4ec-7bf2-49c8-9dce-f229d1fcea8f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:50.128142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1861863-314e-4fb1-9151-8b3e971f67f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:50.128742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1861863-314e-4fb1-9151-8b3e971f67f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.129055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1861863-314e-4fb1-9151-8b3e971f67f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.129603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1861863-314e-4fb1-9151-8b3e971f67f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.139635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1861863-314e-4fb1-9151-8b3e971f67f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:50.141144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1861863-314e-4fb1-9151-8b3e971f67f7 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:50.145294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:50.152785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 943/1928] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:50.236281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd83a8f-1ea1-4f51-8cd0-7b0bcd803219 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.237728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0cd83a8f-1ea1-4f51-8cd0-7b0bcd803219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:50.250183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd83a8f-1ea1-4f51-8cd0-7b0bcd803219 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:50.255409 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:50.256079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.256356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.256623 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] PATH_INFO: `/v3/users/8f4de098424a40f4a384786966abdd7d` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.258012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f4de098424a40f4a384786966abdd7d)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:50.259664 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:50.319656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f4de098424a40f4a384786966abdd7d'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:50.320353 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f4de098424a40f4a384786966abdd7d'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:50.330361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f4de098424a40f4a384786966abdd7d'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:50.331109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f4de098424a40f4a384786966abdd7d'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:50.344784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ad8e6c7f-7f9d-405b-a33c-d5a100ec08a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f4de098424a40f4a384786966abdd7d'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:50.354747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1861863-314e-4fb1-9151-8b3e971f67f7 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.357127 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d1861863-314e-4fb1-9151-8b3e971f67f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:50.371619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 986/1929] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:56:50 2022] DELETE /identity/v3/users/8f4de098424a40f4a384786966abdd7d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:50.375375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1861863-314e-4fb1-9151-8b3e971f67f7 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:50.376741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429a972-2b06-4a1a-9e6c-62cbd31a3cf1 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.377019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429a972-2b06-4a1a-9e6c-62cbd31a3cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.377290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429a972-2b06-4a1a-9e6c-62cbd31a3cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.400191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-a429a972-2b06-4a1a-9e6c-62cbd31a3cf1 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:50.401415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a429a972-2b06-4a1a-9e6c-62cbd31a3cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:50.412403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:50.440585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:50.441302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.441690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.442046 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] PATH_INFO: `/v3/projects/e9854190f10741bc8636cec4604943d5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.443369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9854190f10741bc8636cec4604943d5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:50.445613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:50.474060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 987/1930] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:50.485075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429a972-2b06-4a1a-9e6c-62cbd31a3cf1 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.486409 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a429a972-2b06-4a1a-9e6c-62cbd31a3cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:50.498685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9854190f10741bc8636cec4604943d5', 'request_id': 'req-8a16b351-ca58-4ff4-9567-f2305acec376'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:50.504201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9854190f10741bc8636cec4604943d5', 'request_id': 'req-8a16b351-ca58-4ff4-9567-f2305acec376'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:50.511924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9854190f10741bc8636cec4604943d5', 'request_id': 'req-8a16b351-ca58-4ff4-9567-f2305acec376'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:50.529890 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] The token cache is being invalidate because project e9854190f10741bc8636cec4604943d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:50.530178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:50.530838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a16b351-ca58-4ff4-9567-f2305acec376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:50.532381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 944/1931] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:50 2022] DELETE /identity/v3/projects/e9854190f10741bc8636cec4604943d5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:50.537469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08668620-33f8-4e26-94ed-61b493750152 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.537752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08668620-33f8-4e26-94ed-61b493750152 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.538016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08668620-33f8-4e26-94ed-61b493750152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.552868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429a972-2b06-4a1a-9e6c-62cbd31a3cf1 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:50.563658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-08668620-33f8-4e26-94ed-61b493750152 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:50.564906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-08668620-33f8-4e26-94ed-61b493750152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:50.609687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8c372-c0c7-4e85-bb34-8625e543df23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:50.610683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8c372-c0c7-4e85-bb34-8625e543df23 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.610954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8c372-c0c7-4e85-bb34-8625e543df23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.611215 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8c372-c0c7-4e85-bb34-8625e543df23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.613243 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-12c8c372-c0c7-4e85-bb34-8625e543df23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:50.638879 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 945/1932] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:50.647250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d278b4-7314-4a29-9e6e-1966524d7c4b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.647618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d278b4-7314-4a29-9e6e-1966524d7c4b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.648267 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d278b4-7314-4a29-9e6e-1966524d7c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.669844 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-59d278b4-7314-4a29-9e6e-1966524d7c4b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:50.671087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-59d278b4-7314-4a29-9e6e-1966524d7c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:50.679772 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 946/1933] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:56:50.679951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8c372-c0c7-4e85-bb34-8625e543df23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:50.681508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8c372-c0c7-4e85-bb34-8625e543df23 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:50.683593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d278b4-7314-4a29-9e6e-1966524d7c4b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.692447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 988/1934] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:50.693644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d278b4-7314-4a29-9e6e-1966524d7c4b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:50.698139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59376ef5-6caf-4013-b9e2-b24e2c147399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:50.698732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59376ef5-6caf-4013-b9e2-b24e2c147399 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.699006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59376ef5-6caf-4013-b9e2-b24e2c147399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.699271 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59376ef5-6caf-4013-b9e2-b24e2c147399 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.699932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59376ef5-6caf-4013-b9e2-b24e2c147399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:56:50.700845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59376ef5-6caf-4013-b9e2-b24e2c147399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:50.704685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59376ef5-6caf-4013-b9e2-b24e2c147399 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:50.719010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 947/1935] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:56:50.723357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8c372-c0c7-4e85-bb34-8625e543df23 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.733288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8c372-c0c7-4e85-bb34-8625e543df23 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:50.738027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442470b-b67d-4260-b43c-225ea64c15ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:50.738635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442470b-b67d-4260-b43c-225ea64c15ee admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.738910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442470b-b67d-4260-b43c-225ea64c15ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.739182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442470b-b67d-4260-b43c-225ea64c15ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.739812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442470b-b67d-4260-b43c-225ea64c15ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:50.743293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442470b-b67d-4260-b43c-225ea64c15ee admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:50.766743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:50.774763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 989/1936] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:56:50 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:56:50.779581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59376ef5-6caf-4013-b9e2-b24e2c147399 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.790323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59376ef5-6caf-4013-b9e2-b24e2c147399 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:50.796799 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:50.852832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f407-1d37-4aaf-b5c3-e7b769dab2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:50.853487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f407-1d37-4aaf-b5c3-e7b769dab2ac admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.853845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f407-1d37-4aaf-b5c3-e7b769dab2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.854117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f407-1d37-4aaf-b5c3-e7b769dab2ac admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.854740 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459f407-1d37-4aaf-b5c3-e7b769dab2ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:50.856715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459f407-1d37-4aaf-b5c3-e7b769dab2ac admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:50.881166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 948/1937] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:56:50 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:56:50.885163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442470b-b67d-4260-b43c-225ea64c15ee admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.897549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442470b-b67d-4260-b43c-225ea64c15ee admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:50.902193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d731b0-0525-4189-8efd-43424ee10125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:50.902795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d731b0-0525-4189-8efd-43424ee10125 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.903070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d731b0-0525-4189-8efd-43424ee10125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.903383 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d731b0-0525-4189-8efd-43424ee10125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.904772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d731b0-0525-4189-8efd-43424ee10125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:50.906373 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d731b0-0525-4189-8efd-43424ee10125 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:50.917453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 990/1938] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:56:50.922059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f407-1d37-4aaf-b5c3-e7b769dab2ac admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.932717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f407-1d37-4aaf-b5c3-e7b769dab2ac admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:50.938126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f65d1-e4bd-4124-884e-7f2aadc80610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:50.938800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f65d1-e4bd-4124-884e-7f2aadc80610 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.939080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f65d1-e4bd-4124-884e-7f2aadc80610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.939347 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f65d1-e4bd-4124-884e-7f2aadc80610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.940019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f65d1-e4bd-4124-884e-7f2aadc80610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:50.941730 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f65d1-e4bd-4124-884e-7f2aadc80610 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:50.953239 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 949/1939] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:56:50.957923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d731b0-0525-4189-8efd-43424ee10125 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.970214 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d731b0-0525-4189-8efd-43424ee10125 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:50.979684 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece34e0d-8385-40ce-8df5-b8d1ef8e98f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:50.980045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f65d1-e4bd-4124-884e-7f2aadc80610 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:50.980412 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece34e0d-8385-40ce-8df5-b8d1ef8e98f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:50.980761 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece34e0d-8385-40ce-8df5-b8d1ef8e98f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:50.981033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece34e0d-8385-40ce-8df5-b8d1ef8e98f0 admin admin] PATH_INFO: `/v3/projects/19f786f7363e44d78e478f2fec202f4d/users/1a23bceab9e14ee8907eef2a2df85799/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:50.981975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c45f65d1-e4bd-4124-884e-7f2aadc80610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:50.998780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece34e0d-8385-40ce-8df5-b8d1ef8e98f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f786f7363e44d78e478f2fec202f4d, user_id=1a23bceab9e14ee8907eef2a2df85799, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:50.999559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f65d1-e4bd-4124-884e-7f2aadc80610 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:51.001754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece34e0d-8385-40ce-8df5-b8d1ef8e98f0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:51.004488 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:51.005272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:51.005521 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:51.005776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] PATH_INFO: `/v3/projects/c66687b1c51d42ca9cc0025c29cf1e3a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:51.007265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c66687b1c51d42ca9cc0025c29cf1e3a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:51.010503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:51.019356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 991/1940] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:56:50 2022] PUT /identity/v3/projects/19f786f7363e44d78e478f2fec202f4d/users/1a23bceab9e14ee8907eef2a2df85799/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:56:51.024357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294ce71-2fba-4761-b5e3-1b6526b0b9eb None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:51.024644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294ce71-2fba-4761-b5e3-1b6526b0b9eb None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:51.024970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294ce71-2fba-4761-b5e3-1b6526b0b9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:51.045415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2294ce71-2fba-4761-b5e3-1b6526b0b9eb None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:51.046617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2294ce71-2fba-4761-b5e3-1b6526b0b9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:51.061640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c66687b1c51d42ca9cc0025c29cf1e3a', 'request_id': 'req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:51.068065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c66687b1c51d42ca9cc0025c29cf1e3a', 'request_id': 'req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:51.076588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c66687b1c51d42ca9cc0025c29cf1e3a', 'request_id': 'req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:51.088270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] The token cache is being invalidate because project c66687b1c51d42ca9cc0025c29cf1e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:56:51.088556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:51.089210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:56:51.090943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 950/1941] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:56:50 2022] DELETE /identity/v3/projects/c66687b1c51d42ca9cc0025c29cf1e3a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:56:51.115139 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 992/1942] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:51.127143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:51.128822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:51.195293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b699584-6ee2-4c4b-9f31-0cd88f698e5b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:51.199163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5985c56-c14b-4b71-8b08-0e66973c7022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:51.199947 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5985c56-c14b-4b71-8b08-0e66973c7022 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:51.200357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5985c56-c14b-4b71-8b08-0e66973c7022 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:51.200812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5985c56-c14b-4b71-8b08-0e66973c7022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:51.212442 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5985c56-c14b-4b71-8b08-0e66973c7022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:51.213907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5985c56-c14b-4b71-8b08-0e66973c7022 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:51.226344 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 951/1943] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:51.947481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c818ab5-aaa0-425d-bc56-363682a0fb83 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:51.947760 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c818ab5-aaa0-425d-bc56-363682a0fb83 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:51.948058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c818ab5-aaa0-425d-bc56-363682a0fb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:51.987186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-6c818ab5-aaa0-425d-bc56-363682a0fb83 None None] MFA Rules not processed for user `1a23bceab9e14ee8907eef2a2df85799`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:51.988466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6c818ab5-aaa0-425d-bc56-363682a0fb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:52.059482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 993/1944] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:56:52.071727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5985c56-c14b-4b71-8b08-0e66973c7022 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:56:52.073213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d5985c56-c14b-4b71-8b08-0e66973c7022 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:52.089190 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:56:52.137475 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5985c56-c14b-4b71-8b08-0e66973c7022 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:56:52.141849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a499c03-3a0e-4178-9f0b-c768f6dcf0f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:56:52.142449 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a499c03-3a0e-4178-9f0b-c768f6dcf0f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:52.142721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a499c03-3a0e-4178-9f0b-c768f6dcf0f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:52.142985 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a499c03-3a0e-4178-9f0b-c768f6dcf0f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:52.151588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a499c03-3a0e-4178-9f0b-c768f6dcf0f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:56:52.153067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a499c03-3a0e-4178-9f0b-c768f6dcf0f5 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:56:52.163670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 952/1945] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:56:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:56:53.548388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c4a96-9d0a-4534-953a-5ec040265224 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:56:53.548887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c4a96-9d0a-4534-953a-5ec040265224 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:56:53.548976 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c4a96-9d0a-4534-953a-5ec040265224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:56:53.568282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-5b7c4a96-9d0a-4534-953a-5ec040265224 None None] MFA Rules not processed for user `1a23bceab9e14ee8907eef2a2df85799`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:56:53.569517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5b7c4a96-9d0a-4534-953a-5ec040265224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:56:53.577191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 994/1946] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:56:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:02.159129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=20 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:03.562986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:04.261916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a499c03-3a0e-4178-9f0b-c768f6dcf0f5 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:04.263503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0a499c03-3a0e-4178-9f0b-c768f6dcf0f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:04.279677 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a499c03-3a0e-4178-9f0b-c768f6dcf0f5 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:04.284191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:04.284810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:04.285081 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:04.285503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] PATH_INFO: `/v3/users/d0acb86eeea9488ab2cf6d987e972010` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:04.286795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0acb86eeea9488ab2cf6d987e972010)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:04.289117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:04.340322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0acb86eeea9488ab2cf6d987e972010'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:57:04.340901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0acb86eeea9488ab2cf6d987e972010'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:57:04.348136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0acb86eeea9488ab2cf6d987e972010'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:57:04.348850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0acb86eeea9488ab2cf6d987e972010'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:57:04.361129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0acb86eeea9488ab2cf6d987e972010'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:57:04.376185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 953/1947] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:57:04 2022] DELETE /identity/v3/users/d0acb86eeea9488ab2cf6d987e972010 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:57:04.380679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48d40d-2a46-4859-9091-88a9cc993935 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:04.380955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48d40d-2a46-4859-9091-88a9cc993935 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:04.381216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48d40d-2a46-4859-9091-88a9cc993935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:04.402155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ae48d40d-2a46-4859-9091-88a9cc993935 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:04.403337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ae48d40d-2a46-4859-9091-88a9cc993935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:04.463428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 995/1948] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:04.473370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:04.474803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:04.534479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9481b9-b9c6-40b4-b516-eabeb82b1498 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:04.538367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500390d-86ba-46ae-9ff0-99aab00cca98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:04.538974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500390d-86ba-46ae-9ff0-99aab00cca98 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:04.539256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500390d-86ba-46ae-9ff0-99aab00cca98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:04.539523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500390d-86ba-46ae-9ff0-99aab00cca98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:04.549837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500390d-86ba-46ae-9ff0-99aab00cca98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:04.551464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500390d-86ba-46ae-9ff0-99aab00cca98 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:04.562220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 954/1949] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:57:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:57:04.845638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48d40d-2a46-4859-9091-88a9cc993935 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:04.847158 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ae48d40d-2a46-4859-9091-88a9cc993935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:04.865036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48d40d-2a46-4859-9091-88a9cc993935 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:04.870495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:04.871328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:04.871764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:04.872216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] PATH_INFO: `/v3/projects/2508480db6a848fe8bc4dae99c55bb9c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:04.874073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2508480db6a848fe8bc4dae99c55bb9c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:04.876058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:04.944904 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2508480db6a848fe8bc4dae99c55bb9c', 'request_id': 'req-c36bd83c-a687-4830-b7eb-76ce674ed023'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:57:04.951203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2508480db6a848fe8bc4dae99c55bb9c', 'request_id': 'req-c36bd83c-a687-4830-b7eb-76ce674ed023'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:57:04.961191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2508480db6a848fe8bc4dae99c55bb9c', 'request_id': 'req-c36bd83c-a687-4830-b7eb-76ce674ed023'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:57:04.976910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] The token cache is being invalidate because project 2508480db6a848fe8bc4dae99c55bb9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:57:04.977273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:57:04.977872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c36bd83c-a687-4830-b7eb-76ce674ed023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:57:04.979391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 996/1950] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:57:04 2022] DELETE /identity/v3/projects/2508480db6a848fe8bc4dae99c55bb9c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:57:04.985365 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336478-6867-4c7b-8a99-10852f9a447f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:04.985865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336478-6867-4c7b-8a99-10852f9a447f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:04.986139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336478-6867-4c7b-8a99-10852f9a447f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:05.011263 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-08336478-6867-4c7b-8a99-10852f9a447f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:05.012616 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-08336478-6867-4c7b-8a99-10852f9a447f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:05.085019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 955/1951] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:05.092710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459abb4-cc3e-473e-9f92-dc49a808f035 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:05.092994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459abb4-cc3e-473e-9f92-dc49a808f035 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:05.093267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459abb4-cc3e-473e-9f92-dc49a808f035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:05.115124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4459abb4-cc3e-473e-9f92-dc49a808f035 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:05.116416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4459abb4-cc3e-473e-9f92-dc49a808f035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:05.129853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 997/1952] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:05.131757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336478-6867-4c7b-8a99-10852f9a447f None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:05.142394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336478-6867-4c7b-8a99-10852f9a447f None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:05.147297 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b6166-680d-432f-878a-b842e214dce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:05.147943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b6166-680d-432f-878a-b842e214dce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:05.148226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b6166-680d-432f-878a-b842e214dce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:05.148495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b6166-680d-432f-878a-b842e214dce9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:05.149139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b6166-680d-432f-878a-b842e214dce9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:57:05.149443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b6166-680d-432f-878a-b842e214dce9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:05.151034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b6166-680d-432f-878a-b842e214dce9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:05.165213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 956/1953] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:57:05.173145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459abb4-cc3e-473e-9f92-dc49a808f035 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:05.182330 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459abb4-cc3e-473e-9f92-dc49a808f035 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:05.187642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b9071-0538-462e-a124-f304548a235e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:05.188268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b9071-0538-462e-a124-f304548a235e admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:05.188545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b9071-0538-462e-a124-f304548a235e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:05.188815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b9071-0538-462e-a124-f304548a235e admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:05.189437 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b9071-0538-462e-a124-f304548a235e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:05.191353 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b9071-0538-462e-a124-f304548a235e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:05.219693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 998/1954] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:57:05 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:57:05.223872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b6166-680d-432f-878a-b842e214dce9 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:05.229339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:05.236229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b6166-680d-432f-878a-b842e214dce9 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:05.293561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf29f59-8cdf-495c-8530-90a3ba18ff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:05.294213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf29f59-8cdf-495c-8530-90a3ba18ff52 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:05.294426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf29f59-8cdf-495c-8530-90a3ba18ff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:05.294691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf29f59-8cdf-495c-8530-90a3ba18ff52 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:05.295812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf29f59-8cdf-495c-8530-90a3ba18ff52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:05.298320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf29f59-8cdf-495c-8530-90a3ba18ff52 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:05.319834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 957/1955] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:57:05 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:57:05.323713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b9071-0538-462e-a124-f304548a235e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:05.334744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b9071-0538-462e-a124-f304548a235e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:05.339380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bcd26-9146-4b41-965c-b32435b2d58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:05.340018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bcd26-9146-4b41-965c-b32435b2d58f admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:05.340292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bcd26-9146-4b41-965c-b32435b2d58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:05.340555 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bcd26-9146-4b41-965c-b32435b2d58f admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:05.341215 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bcd26-9146-4b41-965c-b32435b2d58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:05.344191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bcd26-9146-4b41-965c-b32435b2d58f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:05.358331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 999/1956] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:57:05.362823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf29f59-8cdf-495c-8530-90a3ba18ff52 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:05.372825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf29f59-8cdf-495c-8530-90a3ba18ff52 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:05.377968 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38895f98-4a5d-460b-bb0d-3f181f8451d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:05.378570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38895f98-4a5d-460b-bb0d-3f181f8451d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:05.378842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38895f98-4a5d-460b-bb0d-3f181f8451d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:05.379112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38895f98-4a5d-460b-bb0d-3f181f8451d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:05.379744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38895f98-4a5d-460b-bb0d-3f181f8451d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:05.381350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38895f98-4a5d-460b-bb0d-3f181f8451d3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:05.392528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 958/1957] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:57:05.396583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bcd26-9146-4b41-965c-b32435b2d58f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:05.407309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bcd26-9146-4b41-965c-b32435b2d58f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:05.412185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e6dff-100d-4f14-90cb-4a1a13d6f636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:05.412869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e6dff-100d-4f14-90cb-4a1a13d6f636 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:05.413141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e6dff-100d-4f14-90cb-4a1a13d6f636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:05.413427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e6dff-100d-4f14-90cb-4a1a13d6f636 admin admin] PATH_INFO: `/v3/projects/f41c0e3b1239415ba4105ca5014cbe28/users/01e84d6714b34da89292b2246b04d404/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:05.431326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e6dff-100d-4f14-90cb-4a1a13d6f636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41c0e3b1239415ba4105ca5014cbe28, user_id=01e84d6714b34da89292b2246b04d404, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:05.433924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e6dff-100d-4f14-90cb-4a1a13d6f636 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:05.446713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1000/1958] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:57:05 2022] PUT /identity/v3/projects/f41c0e3b1239415ba4105ca5014cbe28/users/01e84d6714b34da89292b2246b04d404/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:57:05.451832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03ec27-17f5-4acf-864f-fd23e5d68e16 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:05.452157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03ec27-17f5-4acf-864f-fd23e5d68e16 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:05.452428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03ec27-17f5-4acf-864f-fd23e5d68e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:05.471682 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-df03ec27-17f5-4acf-864f-fd23e5d68e16 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:05.473470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-df03ec27-17f5-4acf-864f-fd23e5d68e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:05.533525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 959/1959] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:05.543606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e6dff-100d-4f14-90cb-4a1a13d6f636 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:05.546140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-dc4e6dff-100d-4f14-90cb-4a1a13d6f636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:05.568626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:05.605347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e6dff-100d-4f14-90cb-4a1a13d6f636 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:05.609720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6b9e1-37b7-427f-b62c-39eaddd85deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:05.610317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6b9e1-37b7-427f-b62c-39eaddd85deb service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:05.610589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6b9e1-37b7-427f-b62c-39eaddd85deb service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:05.610856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6b9e1-37b7-427f-b62c-39eaddd85deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:05.619650 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6b9e1-37b7-427f-b62c-39eaddd85deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:05.621101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6b9e1-37b7-427f-b62c-39eaddd85deb service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:05.632301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1001/1960] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:57:11.599089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a50f7-cbed-4de7-a736-da9436735c76 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:11.599661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a50f7-cbed-4de7-a736-da9436735c76 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:11.599857 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a50f7-cbed-4de7-a736-da9436735c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:11.629753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-873a50f7-cbed-4de7-a736-da9436735c76 None None] MFA Rules not processed for user `01e84d6714b34da89292b2246b04d404`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:11.631105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-873a50f7-cbed-4de7-a736-da9436735c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:11.698590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 960/1961] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:57:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:11.703867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6b9e1-37b7-427f-b62c-39eaddd85deb service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:11.714482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6b9e1-37b7-427f-b62c-39eaddd85deb service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:11.720011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb5425-1913-4f70-b39a-ad0e707f0068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:11.720619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5425-1913-4f70-b39a-ad0e707f0068 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:11.721052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5425-1913-4f70-b39a-ad0e707f0068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:11.721318 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5425-1913-4f70-b39a-ad0e707f0068 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:11.721961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb5425-1913-4f70-b39a-ad0e707f0068 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:11.723889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb5425-1913-4f70-b39a-ad0e707f0068 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:11.752917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1002/1962] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:57:11 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:57:11.756934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a50f7-cbed-4de7-a736-da9436735c76 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:11.769163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a50f7-cbed-4de7-a736-da9436735c76 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:11.779675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:11.830447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e92b8c-5917-4bbd-a1f3-4f8354c4896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:11.831083 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e92b8c-5917-4bbd-a1f3-4f8354c4896d admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:11.831338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e92b8c-5917-4bbd-a1f3-4f8354c4896d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:11.831605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e92b8c-5917-4bbd-a1f3-4f8354c4896d admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:11.832276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e92b8c-5917-4bbd-a1f3-4f8354c4896d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:11.834188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e92b8c-5917-4bbd-a1f3-4f8354c4896d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:11.858292 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 961/1963] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:57:11 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:57:11.861938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb5425-1913-4f70-b39a-ad0e707f0068 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:11.872851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb5425-1913-4f70-b39a-ad0e707f0068 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:11.878020 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712d707-141e-492d-bdbd-d8f2d3784328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:11.878711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712d707-141e-492d-bdbd-d8f2d3784328 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:11.878985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712d707-141e-492d-bdbd-d8f2d3784328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:11.879279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712d707-141e-492d-bdbd-d8f2d3784328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:11.880009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712d707-141e-492d-bdbd-d8f2d3784328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:11.881617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712d707-141e-492d-bdbd-d8f2d3784328 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:11.895735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1003/1964] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:57:11.900009 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e92b8c-5917-4bbd-a1f3-4f8354c4896d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:11.910825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e92b8c-5917-4bbd-a1f3-4f8354c4896d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:11.915735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b532c-5e63-482a-be44-23767cb663c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:11.916518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b532c-5e63-482a-be44-23767cb663c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:11.916800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b532c-5e63-482a-be44-23767cb663c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:11.917070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b532c-5e63-482a-be44-23767cb663c5 admin admin] PATH_INFO: `/v3/projects/8b7c06918a2c4c84ae618127ce8c44c1/users/4f5db8f3b7774bfe8314cbb95b17786a/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:11.939388 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b532c-5e63-482a-be44-23767cb663c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7c06918a2c4c84ae618127ce8c44c1, user_id=4f5db8f3b7774bfe8314cbb95b17786a, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:11.942394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b532c-5e63-482a-be44-23767cb663c5 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:11.956729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 962/1965] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:57:11 2022] PUT /identity/v3/projects/8b7c06918a2c4c84ae618127ce8c44c1/users/4f5db8f3b7774bfe8314cbb95b17786a/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:57:11.960704 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712d707-141e-492d-bdbd-d8f2d3784328 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:11.971897 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712d707-141e-492d-bdbd-d8f2d3784328 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:12.036587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1b5d8-29b4-4f1d-9379-60536db483a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:12.037203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1b5d8-29b4-4f1d-9379-60536db483a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:12.037482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1b5d8-29b4-4f1d-9379-60536db483a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:12.037749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1b5d8-29b4-4f1d-9379-60536db483a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:12.038391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1b5d8-29b4-4f1d-9379-60536db483a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:12.040325 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1b5d8-29b4-4f1d-9379-60536db483a7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:12.052418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1004/1966] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 17:57:12.056249 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b532c-5e63-482a-be44-23767cb663c5 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:12.067461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b532c-5e63-482a-be44-23767cb663c5 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:12.072561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e805433-eea7-4a57-941c-2c00afde3d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:12.073293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e805433-eea7-4a57-941c-2c00afde3d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:12.073629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e805433-eea7-4a57-941c-2c00afde3d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:12.073963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e805433-eea7-4a57-941c-2c00afde3d84 admin admin] PATH_INFO: `/v3/projects/8b7c06918a2c4c84ae618127ce8c44c1/users/4f5db8f3b7774bfe8314cbb95b17786a/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:12.076048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e805433-eea7-4a57-941c-2c00afde3d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7c06918a2c4c84ae618127ce8c44c1, user_id=4f5db8f3b7774bfe8314cbb95b17786a, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:12.078956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e805433-eea7-4a57-941c-2c00afde3d84 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:12.091699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 963/1967] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:57:12 2022] PUT /identity/v3/projects/8b7c06918a2c4c84ae618127ce8c44c1/users/4f5db8f3b7774bfe8314cbb95b17786a/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:57:17.754454 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c5fea-dde9-45c5-b0be-70b816620869 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:17.755021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c5fea-dde9-45c5-b0be-70b816620869 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:17.755021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c5fea-dde9-45c5-b0be-70b816620869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:17.788102 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2c8c5fea-dde9-45c5-b0be-70b816620869 None None] MFA Rules not processed for user `c091bf027839447da4c713deb291ed1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:17.788828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2c8c5fea-dde9-45c5-b0be-70b816620869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:17.794169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7aabbc-e1c8-41d9-9ca9-b9202204205b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:17.794648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7aabbc-e1c8-41d9-9ca9-b9202204205b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:17.794846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7aabbc-e1c8-41d9-9ca9-b9202204205b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:17.833216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-7b7aabbc-e1c8-41d9-9ca9-b9202204205b None None] MFA Rules not processed for user `4f5db8f3b7774bfe8314cbb95b17786a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:17.834555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7b7aabbc-e1c8-41d9-9ca9-b9202204205b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:17.857619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1005/1968] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:57:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:17.926428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 964/1969] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:57:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:17.939806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c5fea-dde9-45c5-b0be-70b816620869 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:17.941703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2c8c5fea-dde9-45c5-b0be-70b816620869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:17.981214 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:18.011072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c5fea-dde9-45c5-b0be-70b816620869 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:18.015079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a1e7a-f336-48c1-988d-60d31134dcb5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3566d85c9c4c559ba672269ab38f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:18.015814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a1e7a-f336-48c1-988d-60d31134dcb5 service glance] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:18.016185 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a1e7a-f336-48c1-988d-60d31134dcb5 service glance] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:18.016525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a1e7a-f336-48c1-988d-60d31134dcb5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:18.086078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a1e7a-f336-48c1-988d-60d31134dcb5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:18.087813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a1e7a-f336-48c1-988d-60d31134dcb5 service glance] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:18.101323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1006/1970] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:57:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:57:18.316722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593d29d-b45a-442c-925f-827036e8cf5b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:18.317448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593d29d-b45a-442c-925f-827036e8cf5b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:18.317878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593d29d-b45a-442c-925f-827036e8cf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:18.338953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-0593d29d-b45a-442c-925f-827036e8cf5b None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:18.341823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0593d29d-b45a-442c-925f-827036e8cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:18.405433 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:18.419916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 965/1971] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:57:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:18.433741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a1e7a-f336-48c1-988d-60d31134dcb5 service glance] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:18.435378 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3b5a1e7a-f336-48c1-988d-60d31134dcb5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:18.522134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a1e7a-f336-48c1-988d-60d31134dcb5 service glance] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:18.527750 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d532993-68a1-41e6-8405-0c6bed9db986 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:18.528418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d532993-68a1-41e6-8405-0c6bed9db986 service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:18.528693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d532993-68a1-41e6-8405-0c6bed9db986 service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:18.529016 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d532993-68a1-41e6-8405-0c6bed9db986 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:18.543245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d532993-68a1-41e6-8405-0c6bed9db986 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:18.555852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d532993-68a1-41e6-8405-0c6bed9db986 service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:18.555852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1007/1972] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:57:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:57:23.675101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe7e53-cc95-48df-9ad4-c56e1f398733 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:23.675651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe7e53-cc95-48df-9ad4-c56e1f398733 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:23.675651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe7e53-cc95-48df-9ad4-c56e1f398733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:23.697327 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-0ffe7e53-cc95-48df-9ad4-c56e1f398733 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:23.698901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0ffe7e53-cc95-48df-9ad4-c56e1f398733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:23.723525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 966/1973] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:57:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:23.736189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d532993-68a1-41e6-8405-0c6bed9db986 service swift] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:23.747326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d532993-68a1-41e6-8405-0c6bed9db986 service swift] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:23.754173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dec167-b091-4854-b55f-424afd1ca0fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:23.754887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dec167-b091-4854-b55f-424afd1ca0fc service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:23.755224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dec167-b091-4854-b55f-424afd1ca0fc service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:23.755560 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dec167-b091-4854-b55f-424afd1ca0fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:23.769662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dec167-b091-4854-b55f-424afd1ca0fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:23.772757 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dec167-b091-4854-b55f-424afd1ca0fc service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:23.787676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1008/1974] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:57:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:57:31.794695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b646d03-1a7c-4561-87fc-39c9a0dcc37a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:31.795907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b646d03-1a7c-4561-87fc-39c9a0dcc37a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:31.795907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b646d03-1a7c-4561-87fc-39c9a0dcc37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:31.822766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-5b646d03-1a7c-4561-87fc-39c9a0dcc37a None None] MFA Rules not processed for user `01e84d6714b34da89292b2246b04d404`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:31.824231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5b646d03-1a7c-4561-87fc-39c9a0dcc37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:31.836389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 967/1975] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:57:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:33.781869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=72 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:41.816627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=9 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:42.312743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0d5c7-1224-4428-bd51-16d16803c1df None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:42.313784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0d5c7-1224-4428-bd51-16d16803c1df None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:42.314274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0d5c7-1224-4428-bd51-16d16803c1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:42.338603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-fbe0d5c7-1224-4428-bd51-16d16803c1df None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:42.340467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-fbe0d5c7-1224-4428-bd51-16d16803c1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:42.351677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1009/1976] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:57:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:42.365964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b646d03-1a7c-4561-87fc-39c9a0dcc37a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:42.378897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b646d03-1a7c-4561-87fc-39c9a0dcc37a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:42.384324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030892c-2a89-438e-bb97-77c8c0f3c6ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:42.385169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030892c-2a89-438e-bb97-77c8c0f3c6ee service swift] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:42.386117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030892c-2a89-438e-bb97-77c8c0f3c6ee service swift] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:42.386553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030892c-2a89-438e-bb97-77c8c0f3c6ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:42.398094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030892c-2a89-438e-bb97-77c8c0f3c6ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:42.400030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030892c-2a89-438e-bb97-77c8c0f3c6ee service swift] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:42.410281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 968/1977] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:57:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:57:52.332556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:52.405867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=6 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:57:56.784999 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80960665-67de-41b1-a493-24c2123aa3d9 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:56.785568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80960665-67de-41b1-a493-24c2123aa3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:56.785568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80960665-67de-41b1-a493-24c2123aa3d9 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:56.786902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1010/1978] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:57:56.793341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f63e3b-e57a-4233-8915-12370451cbf4 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:56.793814 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f63e3b-e57a-4233-8915-12370451cbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:56.793989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f63e3b-e57a-4233-8915-12370451cbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:56.813598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-60f63e3b-e57a-4233-8915-12370451cbf4 None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:56.814914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-60f63e3b-e57a-4233-8915-12370451cbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:56.884384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 969/1979] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:57:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:57:56.896327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80960665-67de-41b1-a493-24c2123aa3d9 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:56.908329 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80960665-67de-41b1-a493-24c2123aa3d9 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:56.916000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f7b23-7531-4b36-84f3-ade443f9bdef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3566d85c9c4c559ba672269ab38f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:56.916000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7b23-7531-4b36-84f3-ade443f9bdef service glance] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:56.916000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7b23-7531-4b36-84f3-ade443f9bdef service glance] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:56.916000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7b23-7531-4b36-84f3-ade443f9bdef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:56.924854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f7b23-7531-4b36-84f3-ade443f9bdef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:56.926272 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f7b23-7531-4b36-84f3-ade443f9bdef service glance] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:56.936825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1011/1980] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:57:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:57:57.004708 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ae91-438b-4b37-9cc9-daa4cc30ad36 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:57.005033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ae91-438b-4b37-9cc9-daa4cc30ad36 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:57.005329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ae91-438b-4b37-9cc9-daa4cc30ad36 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:57.006699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 970/1981] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:57:57.011387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423da81e-111b-427e-b11c-73a9a2becd96 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:57.011663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423da81e-111b-427e-b11c-73a9a2becd96 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:57.012066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423da81e-111b-427e-b11c-73a9a2becd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:57.034280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-423da81e-111b-427e-b11c-73a9a2becd96 None None] MFA Rules not processed for user `3c17f274e4c14f2f99650844642437a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:57.035463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-423da81e-111b-427e-b11c-73a9a2becd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:57.087571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1012/1982] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:57:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:57.099865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49ae91-438b-4b37-9cc9-daa4cc30ad36 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:57.110465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49ae91-438b-4b37-9cc9-daa4cc30ad36 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:57.113947 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157338d-773f-487b-ab17-c7587ff0b4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:57.114533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157338d-773f-487b-ab17-c7587ff0b4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:57.114807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157338d-773f-487b-ab17-c7587ff0b4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:57.115078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157338d-773f-487b-ab17-c7587ff0b4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:57.122808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157338d-773f-487b-ab17-c7587ff0b4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:57.124996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157338d-773f-487b-ab17-c7587ff0b4c8 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:57.132899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 971/1983] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:57:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:57:57.216056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caf9e6-8bec-4795-8ca6-5b38c56cf2fe None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:57.216335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caf9e6-8bec-4795-8ca6-5b38c56cf2fe None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:57.216601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caf9e6-8bec-4795-8ca6-5b38c56cf2fe None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:57.217970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1013/1984] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:57:57.223454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806eafd-e5b5-499c-8800-091ea4bcbf55 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:57.223732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806eafd-e5b5-499c-8800-091ea4bcbf55 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:57.224029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806eafd-e5b5-499c-8800-091ea4bcbf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:57.244923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-b806eafd-e5b5-499c-8800-091ea4bcbf55 None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:57:57.246163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b806eafd-e5b5-499c-8800-091ea4bcbf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:57:57.256381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 972/1985] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:57:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:57:57.267595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caf9e6-8bec-4795-8ca6-5b38c56cf2fe None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:57:57.278565 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caf9e6-8bec-4795-8ca6-5b38c56cf2fe None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:57:57.282457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29df8d-5b0c-4e90-9e14-fe4d3a270ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:57:57.283054 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29df8d-5b0c-4e90-9e14-fe4d3a270ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:57:57.283327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29df8d-5b0c-4e90-9e14-fe4d3a270ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:57:57.283593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29df8d-5b0c-4e90-9e14-fe4d3a270ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:57:57.294121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29df8d-5b0c-4e90-9e14-fe4d3a270ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:57:57.295556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29df8d-5b0c-4e90-9e14-fe4d3a270ae0 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:57:57.306806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1014/1986] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:57:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:58:01.147573 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d17e-410a-45b9-8b56-cabe39f1bfe0 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:58:01.148150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d17e-410a-45b9-8b56-cabe39f1bfe0 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:58:01.148247 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d17e-410a-45b9-8b56-cabe39f1bfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:58:01.167830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-9c75d17e-410a-45b9-8b56-cabe39f1bfe0 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:58:01.169077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9c75d17e-410a-45b9-8b56-cabe39f1bfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:58:01.176989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 973/1987] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 17:58:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:58:01.200021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29df8d-5b0c-4e90-9e14-fe4d3a270ae0 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:58:01.212939 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29df8d-5b0c-4e90-9e14-fe4d3a270ae0 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:58:01.218006 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b5cd4-97d0-4345-99f8-f1b4f90c61c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:58:01.218656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b5cd4-97d0-4345-99f8-f1b4f90c61c5 service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:58:01.218918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b5cd4-97d0-4345-99f8-f1b4f90c61c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:58:01.219181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b5cd4-97d0-4345-99f8-f1b4f90c61c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:58:01.227598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b5cd4-97d0-4345-99f8-f1b4f90c61c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:58:01.229296 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b5cd4-97d0-4345-99f8-f1b4f90c61c5 service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:58:01.238240 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1015/1988] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:58:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 17:58:10.769670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4b789-97fc-4274-8da7-bd0e33617b7c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:58:10.770291 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4b789-97fc-4274-8da7-bd0e33617b7c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:58:10.770291 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4b789-97fc-4274-8da7-bd0e33617b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:58:10.789653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-2fa4b789-97fc-4274-8da7-bd0e33617b7c None None] MFA Rules not processed for user `1a23bceab9e14ee8907eef2a2df85799`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:58:10.790826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2fa4b789-97fc-4274-8da7-bd0e33617b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:58:10.851449 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 974/1989] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:58:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:58:11.234241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=41 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:58:13.760137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b5cd4-97d0-4345-99f8-f1b4f90c61c5 service swift] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:58:13.761728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ad5b5cd4-97d0-4345-99f8-f1b4f90c61c5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:58:13.843415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b5cd4-97d0-4345-99f8-f1b4f90c61c5 service swift] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:58:13.848762 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb337-93cc-4628-98c9-6ec1636b838c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:58:13.849380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb337-93cc-4628-98c9-6ec1636b838c service placement] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:58:13.849630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb337-93cc-4628-98c9-6ec1636b838c service placement] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:58:13.849896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb337-93cc-4628-98c9-6ec1636b838c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:58:13.851574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a03fb337-93cc-4628-98c9-6ec1636b838c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:58:13.863009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fb337-93cc-4628-98c9-6ec1636b838c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:58:13.864762 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fb337-93cc-4628-98c9-6ec1636b838c service placement] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:58:13.874515 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1016/1990] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:58:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:58:20.843037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=68 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:58:23.870279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=32 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:58:26.159565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be4d08-cd22-454e-99d1-6704a81c5755 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:58:26.160121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be4d08-cd22-454e-99d1-6704a81c5755 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:58:26.160121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be4d08-cd22-454e-99d1-6704a81c5755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:58:26.180911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-44be4d08-cd22-454e-99d1-6704a81c5755 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:58:26.182112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-44be4d08-cd22-454e-99d1-6704a81c5755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:58:26.189171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 975/1991] 10.222.0.27 () {58 vars in 967 bytes} [Fri Oct 14 17:58:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:58:26.201852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb337-93cc-4628-98c9-6ec1636b838c service placement] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:58:26.213899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb337-93cc-4628-98c9-6ec1636b838c service placement] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:58:26.219307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba8d07-ad4a-4566-8864-e5c0546fd099 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:58:26.219938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba8d07-ad4a-4566-8864-e5c0546fd099 service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:58:26.220212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba8d07-ad4a-4566-8864-e5c0546fd099 service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:58:26.220504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba8d07-ad4a-4566-8864-e5c0546fd099 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:58:26.228975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba8d07-ad4a-4566-8864-e5c0546fd099 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:58:26.230532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba8d07-ad4a-4566-8864-e5c0546fd099 service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:58:26.239612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1017/1992] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 17:58:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 17:58:27.617263 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be4d08-cd22-454e-99d1-6704a81c5755 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:58:27.626584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be4d08-cd22-454e-99d1-6704a81c5755 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:58:27.630179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf55b5b-48f1-4228-8a9b-9e5e62631c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:58:27.630772 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf55b5b-48f1-4228-8a9b-9e5e62631c33 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:58:27.631046 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf55b5b-48f1-4228-8a9b-9e5e62631c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:58:27.631313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf55b5b-48f1-4228-8a9b-9e5e62631c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:58:27.640174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf55b5b-48f1-4228-8a9b-9e5e62631c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:58:27.641623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf55b5b-48f1-4228-8a9b-9e5e62631c33 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:58:27.651459 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 976/1993] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:58:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:58:36.235403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=6 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:58:37.647328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=9 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:03.755067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67c420-8052-41ef-8806-4845bd742830 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:03.755598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67c420-8052-41ef-8806-4845bd742830 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:03.755598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67c420-8052-41ef-8806-4845bd742830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:03.777165 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1c67c420-8052-41ef-8806-4845bd742830 None None] MFA Rules not processed for user `c091bf027839447da4c713deb291ed1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:03.778382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1c67c420-8052-41ef-8806-4845bd742830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:03.785310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1018/1994] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:13.770659 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:25.096860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32d50e-c56c-4336-ae56-00211641412c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:25.097357 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32d50e-c56c-4336-ae56-00211641412c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:25.097450 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32d50e-c56c-4336-ae56-00211641412c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:25.116668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-4f32d50e-c56c-4336-ae56-00211641412c None None] MFA Rules not processed for user `b9187955c3cf4b77ba86e67a6f378e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:25.117868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-4f32d50e-c56c-4336-ae56-00211641412c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:25.176033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 977/1995] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:26.355822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c0333-96f6-4a58-8316-e259c97567f6 None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:26.356305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c0333-96f6-4a58-8316-e259c97567f6 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:26.356584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c0333-96f6-4a58-8316-e259c97567f6 None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:26.358642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1019/1996] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 17:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 17:59:26.392761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d015c-47ac-44b2-8c77-bd92fcfae707 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:26.394336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d015c-47ac-44b2-8c77-bd92fcfae707 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:26.394788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d015c-47ac-44b2-8c77-bd92fcfae707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:26.435848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-5b4d015c-47ac-44b2-8c77-bd92fcfae707 None None] MFA Rules not processed for user `73089bc9f85a405b86c4cd3f8cfa830a`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:26.439197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5b4d015c-47ac-44b2-8c77-bd92fcfae707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:26.440044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c0333-96f6-4a58-8316-e259c97567f6 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:26.444255 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-205c0333-96f6-4a58-8316-e259c97567f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:26.458392 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 978/1997] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 17:59:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:26.469433 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d015c-47ac-44b2-8c77-bd92fcfae707 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:26.471049 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5b4d015c-47ac-44b2-8c77-bd92fcfae707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:26.485680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d015c-47ac-44b2-8c77-bd92fcfae707 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:26.492926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff1b73-9c26-488d-8466-262d8f49cd1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:26.493837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff1b73-9c26-488d-8466-262d8f49cd1d service placement] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:26.494166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff1b73-9c26-488d-8466-262d8f49cd1d service placement] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:26.494645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff1b73-9c26-488d-8466-262d8f49cd1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:26.504921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff1b73-9c26-488d-8466-262d8f49cd1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:26.507580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff1b73-9c26-488d-8466-262d8f49cd1d service placement] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:26.528010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 979/1998] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:59:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:59:26.535739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c0333-96f6-4a58-8316-e259c97567f6 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:26.541033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:26.541676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:26.542207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:26.542475 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] PATH_INFO: `/v3/users/1a23bceab9e14ee8907eef2a2df85799` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:26.543787 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a23bceab9e14ee8907eef2a2df85799)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:26.551099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:26.598649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a23bceab9e14ee8907eef2a2df85799'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:26.599313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a23bceab9e14ee8907eef2a2df85799'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:26.606939 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a23bceab9e14ee8907eef2a2df85799'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:26.607578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a23bceab9e14ee8907eef2a2df85799'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:26.619552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-bacc44e9-50c3-4021-89fd-f507372da7df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a23bceab9e14ee8907eef2a2df85799'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:26.632528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1020/1999] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:59:26 2022] DELETE /identity/v3/users/1a23bceab9e14ee8907eef2a2df85799 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:59:26.805481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff1b73-9c26-488d-8466-262d8f49cd1d service placement] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:26.816178 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff1b73-9c26-488d-8466-262d8f49cd1d service placement] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:26.873457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:26.874101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:26.874377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:26.874648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] PATH_INFO: `/v3/projects/19f786f7363e44d78e478f2fec202f4d` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:26.875749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f786f7363e44d78e478f2fec202f4d)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:26.877676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:26.928155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f786f7363e44d78e478f2fec202f4d', 'request_id': 'req-d34f0050-e714-4372-9d15-e011a12b97a4'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:26.934278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f786f7363e44d78e478f2fec202f4d', 'request_id': 'req-d34f0050-e714-4372-9d15-e011a12b97a4'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:26.945247 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f786f7363e44d78e478f2fec202f4d', 'request_id': 'req-d34f0050-e714-4372-9d15-e011a12b97a4'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:26.958132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] The token cache is being invalidate because project 19f786f7363e44d78e478f2fec202f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:59:26.958419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:26.959073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d34f0050-e714-4372-9d15-e011a12b97a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:26.961255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 980/2000] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:59:26 2022] DELETE /identity/v3/projects/19f786f7363e44d78e478f2fec202f4d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:59:26.966272 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f4e6b-b1e0-4041-892c-7538ad67bde3 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:26.966554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f4e6b-b1e0-4041-892c-7538ad67bde3 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:26.966827 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f4e6b-b1e0-4041-892c-7538ad67bde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:26.987076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-509f4e6b-b1e0-4041-892c-7538ad67bde3 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:26.988340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-509f4e6b-b1e0-4041-892c-7538ad67bde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:27.056898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1021/2001] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:27.058665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f6c49-0283-457b-a59a-90258c245999 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:27.058946 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f6c49-0283-457b-a59a-90258c245999 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:27.059220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f6c49-0283-457b-a59a-90258c245999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:27.080018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-493f6c49-0283-457b-a59a-90258c245999 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:27.081345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-493f6c49-0283-457b-a59a-90258c245999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:27.092255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 981/2002] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:27.096304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f4e6b-b1e0-4041-892c-7538ad67bde3 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:27.107346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f4e6b-b1e0-4041-892c-7538ad67bde3 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:27.112051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0421def-def9-4826-9d3f-4a8f8fb12f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:27.112657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0421def-def9-4826-9d3f-4a8f8fb12f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:27.112929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0421def-def9-4826-9d3f-4a8f8fb12f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:27.113194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0421def-def9-4826-9d3f-4a8f8fb12f0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:27.113833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0421def-def9-4826-9d3f-4a8f8fb12f0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:59:27.114127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0421def-def9-4826-9d3f-4a8f8fb12f0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:27.115744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0421def-def9-4826-9d3f-4a8f8fb12f0d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:27.129154 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1022/2003] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 17:59:27.135176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f6c49-0283-457b-a59a-90258c245999 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:27.144567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f6c49-0283-457b-a59a-90258c245999 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:27.150553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286ee36-68ce-4896-9343-de04c5983a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:27.151153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286ee36-68ce-4896-9343-de04c5983a18 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:27.151455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286ee36-68ce-4896-9343-de04c5983a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:27.151725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286ee36-68ce-4896-9343-de04c5983a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:27.152385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286ee36-68ce-4896-9343-de04c5983a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:27.154283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286ee36-68ce-4896-9343-de04c5983a18 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:27.160317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:27.187626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 982/2004] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:59:27 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:59:27.191734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0421def-def9-4826-9d3f-4a8f8fb12f0d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:27.202202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0421def-def9-4826-9d3f-4a8f8fb12f0d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:27.205714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:27.262883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ddd19-a1fd-4a13-a28b-8f3c5039514e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:27.263497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ddd19-a1fd-4a13-a28b-8f3c5039514e admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:27.263774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ddd19-a1fd-4a13-a28b-8f3c5039514e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:27.264089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ddd19-a1fd-4a13-a28b-8f3c5039514e admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:27.264723 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ddd19-a1fd-4a13-a28b-8f3c5039514e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:27.266946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ddd19-a1fd-4a13-a28b-8f3c5039514e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:27.288693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1023/2005] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:59:27 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:59:27.293014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286ee36-68ce-4896-9343-de04c5983a18 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:27.303467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286ee36-68ce-4896-9343-de04c5983a18 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:27.308145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808a9a0-bd5f-4559-9fe7-38dc54a4ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:27.308740 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808a9a0-bd5f-4559-9fe7-38dc54a4ce7d admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:27.309018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808a9a0-bd5f-4559-9fe7-38dc54a4ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:27.309289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808a9a0-bd5f-4559-9fe7-38dc54a4ce7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:27.309926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808a9a0-bd5f-4559-9fe7-38dc54a4ce7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:27.311508 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808a9a0-bd5f-4559-9fe7-38dc54a4ce7d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:27.321983 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 983/2006] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:59:27.325525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ddd19-a1fd-4a13-a28b-8f3c5039514e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:27.336121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ddd19-a1fd-4a13-a28b-8f3c5039514e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:27.340807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7e6b5-df25-4dce-afdf-a794f391dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:27.341398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7e6b5-df25-4dce-afdf-a794f391dbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:27.341675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7e6b5-df25-4dce-afdf-a794f391dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:27.341947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7e6b5-df25-4dce-afdf-a794f391dbe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:27.342627 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7e6b5-df25-4dce-afdf-a794f391dbe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:27.344251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7e6b5-df25-4dce-afdf-a794f391dbe6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:27.355244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1024/2007] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:59:27.359350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808a9a0-bd5f-4559-9fe7-38dc54a4ce7d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:27.369993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808a9a0-bd5f-4559-9fe7-38dc54a4ce7d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:27.374633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bc335-7683-4b9c-a2ad-f1e571e7dd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:27.375279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bc335-7683-4b9c-a2ad-f1e571e7dd3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:27.375578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bc335-7683-4b9c-a2ad-f1e571e7dd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:27.375853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bc335-7683-4b9c-a2ad-f1e571e7dd3a admin admin] PATH_INFO: `/v3/projects/807a7e9d50d44351af16df39c80ad743/users/9e72f38656464f0ba6736dc7a855f3e7/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:27.391349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bc335-7683-4b9c-a2ad-f1e571e7dd3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=807a7e9d50d44351af16df39c80ad743, user_id=9e72f38656464f0ba6736dc7a855f3e7, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:27.394046 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bc335-7683-4b9c-a2ad-f1e571e7dd3a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:27.406880 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 984/2008] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:59:27 2022] PUT /identity/v3/projects/807a7e9d50d44351af16df39c80ad743/users/9e72f38656464f0ba6736dc7a855f3e7/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:59:27.417517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb420c-1a08-49df-af3c-04c99784aa27 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:27.418034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb420c-1a08-49df-af3c-04c99784aa27 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:27.418034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb420c-1a08-49df-af3c-04c99784aa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:27.446742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4bcb420c-1a08-49df-af3c-04c99784aa27 None None] MFA Rules not processed for user `9e72f38656464f0ba6736dc7a855f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:27.447975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4bcb420c-1a08-49df-af3c-04c99784aa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:27.511036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1025/2009] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:27.522100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bc335-7683-4b9c-a2ad-f1e571e7dd3a admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:27.523937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-935bc335-7683-4b9c-a2ad-f1e571e7dd3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:27.587936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bc335-7683-4b9c-a2ad-f1e571e7dd3a admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:27.592267 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0cfef-2feb-4875-b1fb-6326cb905923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:27.592872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0cfef-2feb-4875-b1fb-6326cb905923 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:27.593148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0cfef-2feb-4875-b1fb-6326cb905923 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:27.593417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0cfef-2feb-4875-b1fb-6326cb905923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:27.603378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0cfef-2feb-4875-b1fb-6326cb905923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:27.604848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0cfef-2feb-4875-b1fb-6326cb905923 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:27.615285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 985/2010] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:59:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:59:37.508394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:37.611687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:43.180828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f18395-d4b9-46ca-a5d9-dff6fb86a26d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:43.181364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f18395-d4b9-46ca-a5d9-dff6fb86a26d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:43.181459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f18395-d4b9-46ca-a5d9-dff6fb86a26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:43.201679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-34f18395-d4b9-46ca-a5d9-dff6fb86a26d None None] MFA Rules not processed for user `762e1e38e4234803991187db9b870b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:43.202858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-34f18395-d4b9-46ca-a5d9-dff6fb86a26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:43.256522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1026/2011] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:46.178463 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0cfef-2feb-4875-b1fb-6326cb905923 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:46.270167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0cfef-2feb-4875-b1fb-6326cb905923 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:46.275962 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:46.276687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:46.277052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:46.279034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] PATH_INFO: `/v3/users/9e72f38656464f0ba6736dc7a855f3e7` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:46.279697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e72f38656464f0ba6736dc7a855f3e7)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:46.281742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:46.362582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e72f38656464f0ba6736dc7a855f3e7'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:46.363848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e72f38656464f0ba6736dc7a855f3e7'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:46.373861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e72f38656464f0ba6736dc7a855f3e7'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:46.374579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e72f38656464f0ba6736dc7a855f3e7'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:46.389683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e72f38656464f0ba6736dc7a855f3e7'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:46.407151 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 986/2012] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 17:59:46 2022] DELETE /identity/v3/users/9e72f38656464f0ba6736dc7a855f3e7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:59:46.411401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4c56d-5003-4da0-8f82-d37a4c95924e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:46.411946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4c56d-5003-4da0-8f82-d37a4c95924e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:46.411946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4c56d-5003-4da0-8f82-d37a4c95924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:46.438377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c9e4c56d-5003-4da0-8f82-d37a4c95924e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:46.439588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c9e4c56d-5003-4da0-8f82-d37a4c95924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:46.528115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1027/2013] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:46.543437 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:46.548722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:46.633364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7454d2-5e9a-4ac3-96a6-f5dd8566d14a admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:46.637575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04707b93-a98c-483f-bfc2-14cdb79d1b03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:46.638796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04707b93-a98c-483f-bfc2-14cdb79d1b03 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:46.639085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04707b93-a98c-483f-bfc2-14cdb79d1b03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:46.639360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04707b93-a98c-483f-bfc2-14cdb79d1b03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:46.654503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04707b93-a98c-483f-bfc2-14cdb79d1b03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:46.656656 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04707b93-a98c-483f-bfc2-14cdb79d1b03 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:46.669118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 987/2014] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:59:47.308549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4c56d-5003-4da0-8f82-d37a4c95924e None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:47.310920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c9e4c56d-5003-4da0-8f82-d37a4c95924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:47.334923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4c56d-5003-4da0-8f82-d37a4c95924e None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:47.354991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:47.356116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:47.356629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:47.357144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] PATH_INFO: `/v3/projects/807a7e9d50d44351af16df39c80ad743` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:47.360125 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=807a7e9d50d44351af16df39c80ad743)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:47.363300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:47.453531 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '807a7e9d50d44351af16df39c80ad743', 'request_id': 'req-67a7083e-fd8f-41f2-a7ee-763455948b7c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:47.462503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '807a7e9d50d44351af16df39c80ad743', 'request_id': 'req-67a7083e-fd8f-41f2-a7ee-763455948b7c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:47.474739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '807a7e9d50d44351af16df39c80ad743', 'request_id': 'req-67a7083e-fd8f-41f2-a7ee-763455948b7c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:47.496583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] The token cache is being invalidate because project 807a7e9d50d44351af16df39c80ad743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 17:59:47.496882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:47.497635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-67a7083e-fd8f-41f2-a7ee-763455948b7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 17:59:47.499254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1028/2015] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 17:59:47 2022] DELETE /identity/v3/projects/807a7e9d50d44351af16df39c80ad743 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:59:47.504209 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e02dd-b96c-4b42-a272-f69019059f77 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:47.504667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e02dd-b96c-4b42-a272-f69019059f77 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:47.504758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e02dd-b96c-4b42-a272-f69019059f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:47.531435 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1e0e02dd-b96c-4b42-a272-f69019059f77 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:47.532707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1e0e02dd-b96c-4b42-a272-f69019059f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:47.555732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:47.612674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 988/2016] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 17:59:47.683446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362d696-2697-4baa-8b7e-7b49bdb5ad87 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:47.683721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362d696-2697-4baa-8b7e-7b49bdb5ad87 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:47.684032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362d696-2697-4baa-8b7e-7b49bdb5ad87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:47.704845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1362d696-2697-4baa-8b7e-7b49bdb5ad87 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:47.706066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1362d696-2697-4baa-8b7e-7b49bdb5ad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:47.715227 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1029/2017] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:47.719439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e02dd-b96c-4b42-a272-f69019059f77 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:47.729441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e02dd-b96c-4b42-a272-f69019059f77 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:47.734202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c40b05-00d5-4642-bcba-ec1bdde5172f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:47.734804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c40b05-00d5-4642-bcba-ec1bdde5172f admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:47.735085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c40b05-00d5-4642-bcba-ec1bdde5172f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:47.735359 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c40b05-00d5-4642-bcba-ec1bdde5172f admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:47.736042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c40b05-00d5-4642-bcba-ec1bdde5172f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 17:59:47.736342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c40b05-00d5-4642-bcba-ec1bdde5172f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:47.737935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c40b05-00d5-4642-bcba-ec1bdde5172f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:47.758831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362d696-2697-4baa-8b7e-7b49bdb5ad87 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:47.763389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 989/2018] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 17:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 17:59:47.772823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362d696-2697-4baa-8b7e-7b49bdb5ad87 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:47.777841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777ae3-df96-4f00-a9db-21a6b470f1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:47.778437 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777ae3-df96-4f00-a9db-21a6b470f1c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:47.778705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777ae3-df96-4f00-a9db-21a6b470f1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:47.778965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777ae3-df96-4f00-a9db-21a6b470f1c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:47.779582 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7777ae3-df96-4f00-a9db-21a6b470f1c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:47.781527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7777ae3-df96-4f00-a9db-21a6b470f1c3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:47.803701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:47.814041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1030/2019] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:59:47 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:59:47.818310 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c40b05-00d5-4642-bcba-ec1bdde5172f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:47.830145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c40b05-00d5-4642-bcba-ec1bdde5172f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:47.893379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0295a-59ab-41f6-907f-a00a408679f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:47.894007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0295a-59ab-41f6-907f-a00a408679f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:47.894278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0295a-59ab-41f6-907f-a00a408679f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:47.894536 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0295a-59ab-41f6-907f-a00a408679f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:47.895185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0295a-59ab-41f6-907f-a00a408679f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:47.897145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0295a-59ab-41f6-907f-a00a408679f5 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:47.928664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 990/2020] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:59:47 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:59:47.929797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777ae3-df96-4f00-a9db-21a6b470f1c3 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:47.939992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777ae3-df96-4f00-a9db-21a6b470f1c3 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:47.944519 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287b43d-3a97-488d-b497-363dfe8ae95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:47.945601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287b43d-3a97-488d-b497-363dfe8ae95c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:47.945875 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287b43d-3a97-488d-b497-363dfe8ae95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:47.946144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287b43d-3a97-488d-b497-363dfe8ae95c admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:47.946774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287b43d-3a97-488d-b497-363dfe8ae95c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:47.948398 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287b43d-3a97-488d-b497-363dfe8ae95c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:47.958141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1031/2021] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:59:47.962803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0295a-59ab-41f6-907f-a00a408679f5 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:47.972659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0295a-59ab-41f6-907f-a00a408679f5 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:47.977322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acfaf2-51ab-4686-9643-fdce7f7418e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:47.977916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acfaf2-51ab-4686-9643-fdce7f7418e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:47.978190 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acfaf2-51ab-4686-9643-fdce7f7418e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:47.978460 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acfaf2-51ab-4686-9643-fdce7f7418e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:47.979093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acfaf2-51ab-4686-9643-fdce7f7418e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:47.980719 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acfaf2-51ab-4686-9643-fdce7f7418e5 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:47.991046 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 991/2022] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:59:47.995084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287b43d-3a97-488d-b497-363dfe8ae95c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:48.004845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287b43d-3a97-488d-b497-363dfe8ae95c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:48.010220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01adf4-050f-4e57-afc3-94e55f952811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:48.010852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01adf4-050f-4e57-afc3-94e55f952811 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:48.011125 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01adf4-050f-4e57-afc3-94e55f952811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:48.011397 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01adf4-050f-4e57-afc3-94e55f952811 admin admin] PATH_INFO: `/v3/projects/87b79c338b0047dfb68e9a5b9b16c51b/users/072cedf1a6a24f6fa7d7ed92981344dd/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:48.026387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01adf4-050f-4e57-afc3-94e55f952811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b79c338b0047dfb68e9a5b9b16c51b, user_id=072cedf1a6a24f6fa7d7ed92981344dd, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:48.029068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01adf4-050f-4e57-afc3-94e55f952811 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:48.041563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1032/2023] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:59:47 2022] PUT /identity/v3/projects/87b79c338b0047dfb68e9a5b9b16c51b/users/072cedf1a6a24f6fa7d7ed92981344dd/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:59:48.046757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3a271-d313-43b8-8dd6-a9ce862c331d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:48.047037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3a271-d313-43b8-8dd6-a9ce862c331d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:48.047306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3a271-d313-43b8-8dd6-a9ce862c331d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:48.065710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-b6a3a271-d313-43b8-8dd6-a9ce862c331d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:48.066885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b6a3a271-d313-43b8-8dd6-a9ce862c331d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:48.126168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 992/2024] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:48.136769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01adf4-050f-4e57-afc3-94e55f952811 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:48.138571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ca01adf4-050f-4e57-afc3-94e55f952811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:48.196101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01adf4-050f-4e57-afc3-94e55f952811 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:48.199678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e3d8-0a44-492a-a323-afc49bca054f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:48.200303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e3d8-0a44-492a-a323-afc49bca054f service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:48.200582 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e3d8-0a44-492a-a323-afc49bca054f service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:48.200847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e3d8-0a44-492a-a323-afc49bca054f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:48.209572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719e3d8-0a44-492a-a323-afc49bca054f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:48.211618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719e3d8-0a44-492a-a323-afc49bca054f service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:48.221434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1033/2025] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 17:59:53.245945 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbaa16-03d7-4b48-914e-b2c435b5dabd None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:53.246454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbaa16-03d7-4b48-914e-b2c435b5dabd None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:53.246454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbaa16-03d7-4b48-914e-b2c435b5dabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:53.278626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-58cbaa16-03d7-4b48-914e-b2c435b5dabd None None] MFA Rules not processed for user `072cedf1a6a24f6fa7d7ed92981344dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:53.279850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-58cbaa16-03d7-4b48-914e-b2c435b5dabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:53.299295 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:53.355425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 993/2026] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:53.356658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e3d8-0a44-492a-a323-afc49bca054f service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:53.366889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e3d8-0a44-492a-a323-afc49bca054f service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:53.372589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccbd8c-616c-4457-a38b-60e3e58345ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:53.373182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccbd8c-616c-4457-a38b-60e3e58345ff admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:53.373458 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccbd8c-616c-4457-a38b-60e3e58345ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:53.373732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccbd8c-616c-4457-a38b-60e3e58345ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:53.374365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ccbd8c-616c-4457-a38b-60e3e58345ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:53.376362 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ccbd8c-616c-4457-a38b-60e3e58345ff admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:53.413043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1034/2027] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 17:59:53 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 17:59:53.417732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbaa16-03d7-4b48-914e-b2c435b5dabd None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:53.428795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbaa16-03d7-4b48-914e-b2c435b5dabd None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:53.493575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2feb626-08a7-4bba-9c22-0e053673364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:53.494221 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feb626-08a7-4bba-9c22-0e053673364c admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:53.494495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feb626-08a7-4bba-9c22-0e053673364c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:53.494762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feb626-08a7-4bba-9c22-0e053673364c admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:53.495396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2feb626-08a7-4bba-9c22-0e053673364c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:53.497416 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2feb626-08a7-4bba-9c22-0e053673364c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:53.519911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 994/2028] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 17:59:53 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 17:59:53.523620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccbd8c-616c-4457-a38b-60e3e58345ff admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:53.534069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccbd8c-616c-4457-a38b-60e3e58345ff admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:53.539462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a5965-b7fc-4959-9cd1-00f600f0dae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:53.540078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a5965-b7fc-4959-9cd1-00f600f0dae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:53.540350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a5965-b7fc-4959-9cd1-00f600f0dae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:53.540616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a5965-b7fc-4959-9cd1-00f600f0dae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:53.541262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a5965-b7fc-4959-9cd1-00f600f0dae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:53.542864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a5965-b7fc-4959-9cd1-00f600f0dae5 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:53.554282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1035/2029] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:59:53.559336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2feb626-08a7-4bba-9c22-0e053673364c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:53.571405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2feb626-08a7-4bba-9c22-0e053673364c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:53.576229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d65d6-4a86-474c-8922-c450f870abf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:53.576885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d65d6-4a86-474c-8922-c450f870abf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:53.577163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d65d6-4a86-474c-8922-c450f870abf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:53.577434 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d65d6-4a86-474c-8922-c450f870abf9 admin admin] PATH_INFO: `/v3/projects/97ffbaca123c421f89cfb397545fbe66/users/ad5e8ab4050c4cf0969c0b09ef652544/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:53.593968 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d65d6-4a86-474c-8922-c450f870abf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ffbaca123c421f89cfb397545fbe66, user_id=ad5e8ab4050c4cf0969c0b09ef652544, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:53.596591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d65d6-4a86-474c-8922-c450f870abf9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:53.612650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 995/2030] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:59:53 2022] PUT /identity/v3/projects/97ffbaca123c421f89cfb397545fbe66/users/ad5e8ab4050c4cf0969c0b09ef652544/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 17:59:53.616604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a5965-b7fc-4959-9cd1-00f600f0dae5 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:53.627400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a5965-b7fc-4959-9cd1-00f600f0dae5 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:53.689919 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d508f-4f84-49e5-9f9b-fe51448feb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:53.690535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d508f-4f84-49e5-9f9b-fe51448feb4f admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:53.690809 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d508f-4f84-49e5-9f9b-fe51448feb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:53.691079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d508f-4f84-49e5-9f9b-fe51448feb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:53.691776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d508f-4f84-49e5-9f9b-fe51448feb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:53.693421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d508f-4f84-49e5-9f9b-fe51448feb4f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:53.707230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1036/2031] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 17:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 17:59:53.708331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d65d6-4a86-474c-8922-c450f870abf9 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:53.719602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d65d6-4a86-474c-8922-c450f870abf9 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:53.724200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ea75c-b2b4-43aa-8dcd-cff63d683ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:53.724845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ea75c-b2b4-43aa-8dcd-cff63d683ef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:53.725117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ea75c-b2b4-43aa-8dcd-cff63d683ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:53.725382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ea75c-b2b4-43aa-8dcd-cff63d683ef9 admin admin] PATH_INFO: `/v3/projects/97ffbaca123c421f89cfb397545fbe66/users/ad5e8ab4050c4cf0969c0b09ef652544/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:53.728436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ea75c-b2b4-43aa-8dcd-cff63d683ef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ffbaca123c421f89cfb397545fbe66, user_id=ad5e8ab4050c4cf0969c0b09ef652544, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:53.731090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ea75c-b2b4-43aa-8dcd-cff63d683ef9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:53.741621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 996/2032] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 17:59:53 2022] PUT /identity/v3/projects/97ffbaca123c421f89cfb397545fbe66/users/ad5e8ab4050c4cf0969c0b09ef652544/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 17:59:59.322090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d2e0-933f-4090-b03e-20a091bc1b0a None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:59.322947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d2e0-933f-4090-b03e-20a091bc1b0a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:59.323372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d2e0-933f-4090-b03e-20a091bc1b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:59.389783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-5d82d2e0-933f-4090-b03e-20a091bc1b0a None None] MFA Rules not processed for user `ad5e8ab4050c4cf0969c0b09ef652544`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 17:59:59.393328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5d82d2e0-933f-4090-b03e-20a091bc1b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:59.422450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:59.515680 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1037/2033] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 17:59:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 17:59:59.528690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ea75c-b2b4-43aa-8dcd-cff63d683ef9 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 17:59:59.530795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-644ea75c-b2b4-43aa-8dcd-cff63d683ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 17:59:59.666138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ea75c-b2b4-43aa-8dcd-cff63d683ef9 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 17:59:59.695033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16ce5c-5d17-4273-af05-a529fdab26ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 17:59:59.696719 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16ce5c-5d17-4273-af05-a529fdab26ca service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 17:59:59.703446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16ce5c-5d17-4273-af05-a529fdab26ca service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 17:59:59.704102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16ce5c-5d17-4273-af05-a529fdab26ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 17:59:59.722762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 17:59:59.845948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16ce5c-5d17-4273-af05-a529fdab26ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 17:59:59.849229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16ce5c-5d17-4273-af05-a529fdab26ca service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 17:59:59.864148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 997/2034] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 17:59:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:04.958506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d2e0-933f-4090-b03e-20a091bc1b0a None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:04.960056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5d82d2e0-933f-4090-b03e-20a091bc1b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:05.021599 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d2e0-933f-4090-b03e-20a091bc1b0a None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:05.025698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef753f0e-2e6d-424f-a45c-6d0b866d97c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:05.026296 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef753f0e-2e6d-424f-a45c-6d0b866d97c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:05.026574 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef753f0e-2e6d-424f-a45c-6d0b866d97c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:05.026844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef753f0e-2e6d-424f-a45c-6d0b866d97c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:05.028779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ef753f0e-2e6d-424f-a45c-6d0b866d97c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:05.045215 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef753f0e-2e6d-424f-a45c-6d0b866d97c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:05.046685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef753f0e-2e6d-424f-a45c-6d0b866d97c5 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:05.059651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1038/2035] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:06.247657 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16ce5c-5d17-4273-af05-a529fdab26ca service nova] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:06.256995 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16ce5c-5d17-4273-af05-a529fdab26ca service nova] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:06.260519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e548d-53fa-4b97-b178-de2781f3ca85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:06.261697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e548d-53fa-4b97-b178-de2781f3ca85 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:06.262145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e548d-53fa-4b97-b178-de2781f3ca85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:06.262622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e548d-53fa-4b97-b178-de2781f3ca85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:06.264389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ef6e548d-53fa-4b97-b178-de2781f3ca85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:06.277583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e548d-53fa-4b97-b178-de2781f3ca85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:06.279007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e548d-53fa-4b97-b178-de2781f3ca85 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:06.290937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 998/2036] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:15.053250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=46 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:16.023542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef753f0e-2e6d-424f-a45c-6d0b866d97c5 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:16.034777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef753f0e-2e6d-424f-a45c-6d0b866d97c5 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:16.040538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb535ab-d680-4094-9ac8-ac1a1e127ddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:16.041282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb535ab-d680-4094-9ac8-ac1a1e127ddb service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:16.041565 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb535ab-d680-4094-9ac8-ac1a1e127ddb service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:16.041852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb535ab-d680-4094-9ac8-ac1a1e127ddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:16.043676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3bb535ab-d680-4094-9ac8-ac1a1e127ddb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:16.105579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb535ab-d680-4094-9ac8-ac1a1e127ddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:16.107166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb535ab-d680-4094-9ac8-ac1a1e127ddb service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:16.117699 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1039/2037] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:16.163458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e548d-53fa-4b97-b178-de2781f3ca85 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:16.173036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e548d-53fa-4b97-b178-de2781f3ca85 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:16.176572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05d505-bf5a-4ea9-b130-d4515251d393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:16.177154 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05d505-bf5a-4ea9-b130-d4515251d393 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:16.177423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05d505-bf5a-4ea9-b130-d4515251d393 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:16.177712 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05d505-bf5a-4ea9-b130-d4515251d393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:16.179382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7e05d505-bf5a-4ea9-b130-d4515251d393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:16.247372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05d505-bf5a-4ea9-b130-d4515251d393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:16.249380 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05d505-bf5a-4ea9-b130-d4515251d393 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:16.258902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 999/2038] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:00:18.557345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777a013-6387-486b-8d69-4d4e8b9df305 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:18.558201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777a013-6387-486b-8d69-4d4e8b9df305 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:18.558642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777a013-6387-486b-8d69-4d4e8b9df305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:18.583335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e777a013-6387-486b-8d69-4d4e8b9df305 None None] MFA Rules not processed for user `072cedf1a6a24f6fa7d7ed92981344dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:18.584873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e777a013-6387-486b-8d69-4d4e8b9df305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:18.599550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1040/2039] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:19.206199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05d505-bf5a-4ea9-b130-d4515251d393 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:19.219249 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05d505-bf5a-4ea9-b130-d4515251d393 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:19.223410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170af7c-40cf-404d-9e57-d7ba4fd82c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:19.224189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170af7c-40cf-404d-9e57-d7ba4fd82c4f service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:19.224690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170af7c-40cf-404d-9e57-d7ba4fd82c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:19.225503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170af7c-40cf-404d-9e57-d7ba4fd82c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:19.227549 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7170af7c-40cf-404d-9e57-d7ba4fd82c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:19.238828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170af7c-40cf-404d-9e57-d7ba4fd82c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:19.240429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170af7c-40cf-404d-9e57-d7ba4fd82c4f service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:19.249313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1000/2040] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:20.107034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777a013-6387-486b-8d69-4d4e8b9df305 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:20.110826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e777a013-6387-486b-8d69-4d4e8b9df305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:20.252294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777a013-6387-486b-8d69-4d4e8b9df305 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:20.258729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6097c9-f5c0-46a9-8ab2-f1da90d5251e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:20.259510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6097c9-f5c0-46a9-8ab2-f1da90d5251e service placement] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:20.259918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6097c9-f5c0-46a9-8ab2-f1da90d5251e service placement] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:20.260253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6097c9-f5c0-46a9-8ab2-f1da90d5251e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:20.262233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ad6097c9-f5c0-46a9-8ab2-f1da90d5251e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:20.282435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6097c9-f5c0-46a9-8ab2-f1da90d5251e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:20.285472 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6097c9-f5c0-46a9-8ab2-f1da90d5251e service placement] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:20.310041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1041/2041] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:00:20.342480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170af7c-40cf-404d-9e57-d7ba4fd82c4f service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:20.344051 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7170af7c-40cf-404d-9e57-d7ba4fd82c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:20.371801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170af7c-40cf-404d-9e57-d7ba4fd82c4f service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:20.384594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:20.385238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:20.386003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:20.386478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] PATH_INFO: `/v3/users/c091bf027839447da4c713deb291ed1f` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:20.387800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c091bf027839447da4c713deb291ed1f)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:20.394251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:20.461771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c091bf027839447da4c713deb291ed1f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:20.462433 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c091bf027839447da4c713deb291ed1f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:20.470105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c091bf027839447da4c713deb291ed1f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:20.470768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c091bf027839447da4c713deb291ed1f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:20.481988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:20.485336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-86474fd8-9ce8-4fd0-b749-36834ac89f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c091bf027839447da4c713deb291ed1f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:20.499856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1001/2042] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:00:20 2022] DELETE /identity/v3/users/c091bf027839447da4c713deb291ed1f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:00:20.712239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6097c9-f5c0-46a9-8ab2-f1da90d5251e service placement] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:20.725439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6097c9-f5c0-46a9-8ab2-f1da90d5251e service placement] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:20.798980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:20.799578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:20.799863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:20.800161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] PATH_INFO: `/v3/projects/530fdfc3448a40b7aa85170873d3f0ff` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:20.801279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=530fdfc3448a40b7aa85170873d3f0ff)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:20.803197 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:20.829100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:20.855727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530fdfc3448a40b7aa85170873d3f0ff', 'request_id': 'req-a18c3d8f-8447-496a-b178-739115086749'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:20.862525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530fdfc3448a40b7aa85170873d3f0ff', 'request_id': 'req-a18c3d8f-8447-496a-b178-739115086749'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:20.871034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530fdfc3448a40b7aa85170873d3f0ff', 'request_id': 'req-a18c3d8f-8447-496a-b178-739115086749'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:20.883008 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] The token cache is being invalidate because project 530fdfc3448a40b7aa85170873d3f0ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:00:20.883404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:20.884085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a18c3d8f-8447-496a-b178-739115086749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:20.885570 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1042/2043] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:00:20 2022] DELETE /identity/v3/projects/530fdfc3448a40b7aa85170873d3f0ff => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:00:20.890519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807096b1-c63d-44be-af10-228c348fc375 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:20.890940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807096b1-c63d-44be-af10-228c348fc375 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:20.891273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807096b1-c63d-44be-af10-228c348fc375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:20.912597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-807096b1-c63d-44be-af10-228c348fc375 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:20.913802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-807096b1-c63d-44be-af10-228c348fc375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:20.975165 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1002/2044] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:20.983353 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ab7e8-6bec-4d0d-b4bf-2e66393d39e0 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:20.983637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ab7e8-6bec-4d0d-b4bf-2e66393d39e0 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:20.983950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ab7e8-6bec-4d0d-b4bf-2e66393d39e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.004083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-fc9ab7e8-6bec-4d0d-b4bf-2e66393d39e0 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:21.005400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-fc9ab7e8-6bec-4d0d-b4bf-2e66393d39e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:21.013581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1043/2045] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:21.018426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807096b1-c63d-44be-af10-228c348fc375 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:21.028806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807096b1-c63d-44be-af10-228c348fc375 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:21.034173 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d703f00-a139-48f9-992a-00181251a84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:21.034875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d703f00-a139-48f9-992a-00181251a84c admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:21.035041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d703f00-a139-48f9-992a-00181251a84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:21.035311 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d703f00-a139-48f9-992a-00181251a84c admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.035991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d703f00-a139-48f9-992a-00181251a84c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:00:21.036292 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d703f00-a139-48f9-992a-00181251a84c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:21.037883 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d703f00-a139-48f9-992a-00181251a84c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:21.054781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1003/2046] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:00:21.060085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ab7e8-6bec-4d0d-b4bf-2e66393d39e0 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:21.069453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ab7e8-6bec-4d0d-b4bf-2e66393d39e0 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:21.074618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9d7d8-7415-4fc2-afbc-2574b30c86cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:21.075286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9d7d8-7415-4fc2-afbc-2574b30c86cf admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:21.075562 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9d7d8-7415-4fc2-afbc-2574b30c86cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:21.075868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9d7d8-7415-4fc2-afbc-2574b30c86cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.077027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9d7d8-7415-4fc2-afbc-2574b30c86cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:21.079499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9d7d8-7415-4fc2-afbc-2574b30c86cf admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:21.118954 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6378f2-cedd-47a0-950a-0442626ff795 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:21.119439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6378f2-cedd-47a0-950a-0442626ff795 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:21.119909 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6378f2-cedd-47a0-950a-0442626ff795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.122212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1044/2047] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:00:21 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:00:21.123135 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9d7d8-7415-4fc2-afbc-2574b30c86cf admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:21.146471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-9c6378f2-cedd-47a0-950a-0442626ff795 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:21.147679 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9c6378f2-cedd-47a0-950a-0442626ff795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:21.196051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9d7d8-7415-4fc2-afbc-2574b30c86cf admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:21.200730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa53a3f-d9be-4359-9d67-ae95bc0b96ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:21.201501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa53a3f-d9be-4359-9d67-ae95bc0b96ad admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:21.202870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa53a3f-d9be-4359-9d67-ae95bc0b96ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:21.203181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa53a3f-d9be-4359-9d67-ae95bc0b96ad admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.203828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa53a3f-d9be-4359-9d67-ae95bc0b96ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:21.205888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa53a3f-d9be-4359-9d67-ae95bc0b96ad admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:21.218218 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1004/2048] 10.222.0.27 () {58 vars in 967 bytes} [Fri Oct 14 18:00:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:21.228147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1045/2049] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:00:21 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:00:21.236487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6378f2-cedd-47a0-950a-0442626ff795 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:21.250862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6378f2-cedd-47a0-950a-0442626ff795 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:21.257723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1b48d-b89d-484d-ba1e-5da68a088e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:21.258536 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1b48d-b89d-484d-ba1e-5da68a088e01 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:21.259142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1b48d-b89d-484d-ba1e-5da68a088e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:21.259570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1b48d-b89d-484d-ba1e-5da68a088e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.260678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1b48d-b89d-484d-ba1e-5da68a088e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:21.262321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1b48d-b89d-484d-ba1e-5da68a088e01 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:21.274244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1005/2050] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:00:21.282268 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa53a3f-d9be-4359-9d67-ae95bc0b96ad admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:21.284981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1b48d-b89d-484d-ba1e-5da68a088e01 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:21.286659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b5f1b48d-b89d-484d-ba1e-5da68a088e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:21.293599 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa53a3f-d9be-4359-9d67-ae95bc0b96ad admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:21.300221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439c439-f481-4052-b9a7-ea251d8477f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:21.300815 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439c439-f481-4052-b9a7-ea251d8477f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:21.301092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439c439-f481-4052-b9a7-ea251d8477f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:21.301360 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439c439-f481-4052-b9a7-ea251d8477f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.301998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439c439-f481-4052-b9a7-ea251d8477f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:21.303593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439c439-f481-4052-b9a7-ea251d8477f6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:21.317354 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1046/2051] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:00:21.318304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439c439-f481-4052-b9a7-ea251d8477f6 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:21.328957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439c439-f481-4052-b9a7-ea251d8477f6 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:21.333624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a070c-8320-48d9-85eb-a1c8418d77ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:21.334266 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a070c-8320-48d9-85eb-a1c8418d77ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:21.334540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a070c-8320-48d9-85eb-a1c8418d77ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:21.334813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a070c-8320-48d9-85eb-a1c8418d77ef admin admin] PATH_INFO: `/v3/projects/bfdb342c57b144a384e9938fa4c6e0bf/users/3a0657648d244f12ab76c068b38e641b/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.356099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a070c-8320-48d9-85eb-a1c8418d77ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfdb342c57b144a384e9938fa4c6e0bf, user_id=3a0657648d244f12ab76c068b38e641b, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:21.359309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a070c-8320-48d9-85eb-a1c8418d77ef admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:21.366581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1b48d-b89d-484d-ba1e-5da68a088e01 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:21.372999 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1047/2052] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:00:21 2022] PUT /identity/v3/projects/bfdb342c57b144a384e9938fa4c6e0bf/users/3a0657648d244f12ab76c068b38e641b/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:00:21.375272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:21.378260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2a0aa-4626-46ca-a8ab-ae1e3040555d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:21.378553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2a0aa-4626-46ca-a8ab-ae1e3040555d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:21.378824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2a0aa-4626-46ca-a8ab-ae1e3040555d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.399193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7ac2a0aa-4626-46ca-a8ab-ae1e3040555d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:21.400440 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7ac2a0aa-4626-46ca-a8ab-ae1e3040555d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:21.438270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=6 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:21.446907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e569647-a719-4fc9-abec-876663e13e41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:21.448081 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e569647-a719-4fc9-abec-876663e13e41 service swift] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:21.449082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e569647-a719-4fc9-abec-876663e13e41 service swift] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:21.449681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e569647-a719-4fc9-abec-876663e13e41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.474076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1048/2053] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:00:21.484970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2a0aa-4626-46ca-a8ab-ae1e3040555d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:21.486629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7ac2a0aa-4626-46ca-a8ab-ae1e3040555d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:21.519308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e569647-a719-4fc9-abec-876663e13e41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:21.520530 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e569647-a719-4fc9-abec-876663e13e41 service swift] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:21.530193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1006/2054] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 18:00:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 18:00:21.560556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2a0aa-4626-46ca-a8ab-ae1e3040555d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:21.565188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1ca30-45bf-44bd-924e-47af187311a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:21.566751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1ca30-45bf-44bd-924e-47af187311a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:21.566751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1ca30-45bf-44bd-924e-47af187311a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:21.566751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1ca30-45bf-44bd-924e-47af187311a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:21.577929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1ca30-45bf-44bd-924e-47af187311a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:21.579552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1ca30-45bf-44bd-924e-47af187311a6 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:21.590992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1049/2055] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:29.171159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093b7f6-782a-43a8-b34f-9a83e8a8a83c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:29.172293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093b7f6-782a-43a8-b34f-9a83e8a8a83c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:29.172574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093b7f6-782a-43a8-b34f-9a83e8a8a83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:29.242211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6093b7f6-782a-43a8-b34f-9a83e8a8a83c None None] MFA Rules not processed for user `3a0657648d244f12ab76c068b38e641b`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:29.243574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6093b7f6-782a-43a8-b34f-9a83e8a8a83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:29.369918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1007/2056] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:29.385799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1ca30-45bf-44bd-924e-47af187311a6 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:29.387423 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d8a1ca30-45bf-44bd-924e-47af187311a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:29.530382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1ca30-45bf-44bd-924e-47af187311a6 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:29.536459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbd3b2-7e39-4b6d-a9da-f9d475f20027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:29.537184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbd3b2-7e39-4b6d-a9da-f9d475f20027 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:29.537535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbd3b2-7e39-4b6d-a9da-f9d475f20027 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:29.537881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbd3b2-7e39-4b6d-a9da-f9d475f20027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:29.548517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbd3b2-7e39-4b6d-a9da-f9d475f20027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:29.551703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbd3b2-7e39-4b6d-a9da-f9d475f20027 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:29.580456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1050/2057] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:32.252219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91775c-1714-4725-88e8-19dd39fd312a None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:32.252761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91775c-1714-4725-88e8-19dd39fd312a None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:32.252761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91775c-1714-4725-88e8-19dd39fd312a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:32.290906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-4c91775c-1714-4725-88e8-19dd39fd312a None None] MFA Rules not processed for user `3a0657648d244f12ab76c068b38e641b`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:32.293972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-4c91775c-1714-4725-88e8-19dd39fd312a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:32.304836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1008/2058] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:32.339070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbd3b2-7e39-4b6d-a9da-f9d475f20027 service nova] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:32.341069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-54fbd3b2-7e39-4b6d-a9da-f9d475f20027 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:32.405336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91775c-1714-4725-88e8-19dd39fd312a None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:32.407200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-4c91775c-1714-4725-88e8-19dd39fd312a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:32.432367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91775c-1714-4725-88e8-19dd39fd312a None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:32.453525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:32.454282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:32.454633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:32.454984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] PATH_INFO: `/v3/users/01e84d6714b34da89292b2246b04d404` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:32.458216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbd3b2-7e39-4b6d-a9da-f9d475f20027 service nova] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:32.460930 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e84d6714b34da89292b2246b04d404)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:32.464341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca5993-9b5f-4775-b17d-1734fd830d03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:32.464938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca5993-9b5f-4775-b17d-1734fd830d03 service placement] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:32.465210 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca5993-9b5f-4775-b17d-1734fd830d03 service placement] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:32.465473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca5993-9b5f-4775-b17d-1734fd830d03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:32.468468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:32.470776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-97ca5993-9b5f-4775-b17d-1734fd830d03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:32.486290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:32.490587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca5993-9b5f-4775-b17d-1734fd830d03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:32.494572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca5993-9b5f-4775-b17d-1734fd830d03 service placement] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:32.521490 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:32.536058 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1051/2059] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:32.542527 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e84d6714b34da89292b2246b04d404'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:32.543713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e84d6714b34da89292b2246b04d404'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:32.562606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e84d6714b34da89292b2246b04d404'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:32.564241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e84d6714b34da89292b2246b04d404'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:32.587388 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e84d6714b34da89292b2246b04d404'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:32.607870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1009/2060] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:00:32 2022] DELETE /identity/v3/users/01e84d6714b34da89292b2246b04d404 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:00:32.862247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca5993-9b5f-4775-b17d-1734fd830d03 service placement] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:32.874484 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca5993-9b5f-4775-b17d-1734fd830d03 service placement] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:32.953527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:32.954155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:32.954544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:32.954818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] PATH_INFO: `/v3/projects/f41c0e3b1239415ba4105ca5014cbe28` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:32.956081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41c0e3b1239415ba4105ca5014cbe28)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:32.958263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:33.014811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41c0e3b1239415ba4105ca5014cbe28', 'request_id': 'req-066affe2-8dd1-4c57-87a6-f62a6ba8da31'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.020613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41c0e3b1239415ba4105ca5014cbe28', 'request_id': 'req-066affe2-8dd1-4c57-87a6-f62a6ba8da31'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.029543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41c0e3b1239415ba4105ca5014cbe28', 'request_id': 'req-066affe2-8dd1-4c57-87a6-f62a6ba8da31'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.048298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] The token cache is being invalidate because project f41c0e3b1239415ba4105ca5014cbe28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:00:33.048701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.049449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.051156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1052/2061] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:00:32 2022] DELETE /identity/v3/projects/f41c0e3b1239415ba4105ca5014cbe28 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:00:33.055981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:33.057491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:33.071867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ca0c8-573d-4868-9e39-b7ce9e2e30b8 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:33.133037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:33.133977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:33.134460 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:33.134739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] PATH_INFO: `/v3/users/4f5db8f3b7774bfe8314cbb95b17786a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:33.135926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f5db8f3b7774bfe8314cbb95b17786a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:33.137866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:33.240526 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f5db8f3b7774bfe8314cbb95b17786a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.244045 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:33.245674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:33.253660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f5db8f3b7774bfe8314cbb95b17786a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.272351 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f5db8f3b7774bfe8314cbb95b17786a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.276839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f5db8f3b7774bfe8314cbb95b17786a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.308313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f5db8f3b7774bfe8314cbb95b17786a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.329734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1010/2062] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:00:33 2022] DELETE /identity/v3/users/4f5db8f3b7774bfe8314cbb95b17786a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:00:33.396212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066affe2-8dd1-4c57-87a6-f62a6ba8da31 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:33.434033 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:33.498012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87245465-8c43-4879-8597-394080e7e3f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:33.499161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87245465-8c43-4879-8597-394080e7e3f3 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:33.499441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87245465-8c43-4879-8597-394080e7e3f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:33.499706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87245465-8c43-4879-8597-394080e7e3f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:33.501522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-87245465-8c43-4879-8597-394080e7e3f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:33.543451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:33.544951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:33.581024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87245465-8c43-4879-8597-394080e7e3f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:33.582716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87245465-8c43-4879-8597-394080e7e3f3 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:33.583840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:33.594049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1053/2063] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:00:33.628630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f09bff-c839-4bb5-a6ce-eff2b0185dd7 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:33.635390 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:33.636587 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:33.636587 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:33.636907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] PATH_INFO: `/v3/projects/8b7c06918a2c4c84ae618127ce8c44c1` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:33.638147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b7c06918a2c4c84ae618127ce8c44c1)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:33.640567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:33.701088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b7c06918a2c4c84ae618127ce8c44c1', 'request_id': 'req-28ed9f86-53df-443b-8464-bbb452d1a1fe'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.709757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b7c06918a2c4c84ae618127ce8c44c1', 'request_id': 'req-28ed9f86-53df-443b-8464-bbb452d1a1fe'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.718800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b7c06918a2c4c84ae618127ce8c44c1', 'request_id': 'req-28ed9f86-53df-443b-8464-bbb452d1a1fe'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.733207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] The token cache is being invalidate because project 8b7c06918a2c4c84ae618127ce8c44c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:00:33.733599 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.734366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28ed9f86-53df-443b-8464-bbb452d1a1fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:00:33.736217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1011/2064] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:00:33 2022] DELETE /identity/v3/projects/8b7c06918a2c4c84ae618127ce8c44c1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:00:33.740800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf52961-bc63-4d84-8488-16cce7894d65 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:33.741088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf52961-bc63-4d84-8488-16cce7894d65 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:33.741370 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf52961-bc63-4d84-8488-16cce7894d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:33.761714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-bdf52961-bc63-4d84-8488-16cce7894d65 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:33.762979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-bdf52961-bc63-4d84-8488-16cce7894d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:33.845062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1054/2065] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:33.855550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4722dfd-0cfd-4d3d-aaa3-4099870a74dd None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:33.855836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4722dfd-0cfd-4d3d-aaa3-4099870a74dd None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:33.856176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4722dfd-0cfd-4d3d-aaa3-4099870a74dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:33.879481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c4722dfd-0cfd-4d3d-aaa3-4099870a74dd None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:33.880812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c4722dfd-0cfd-4d3d-aaa3-4099870a74dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:33.889532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1012/2066] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:33.893843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf52961-bc63-4d84-8488-16cce7894d65 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:33.907109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf52961-bc63-4d84-8488-16cce7894d65 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:33.912385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde8bc7-d30c-4663-830e-32c860e03fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:33.913131 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde8bc7-d30c-4663-830e-32c860e03fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:33.913405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde8bc7-d30c-4663-830e-32c860e03fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:33.913741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde8bc7-d30c-4663-830e-32c860e03fc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:33.914470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde8bc7-d30c-4663-830e-32c860e03fc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:00:33.914766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde8bc7-d30c-4663-830e-32c860e03fc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:33.916542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde8bc7-d30c-4663-830e-32c860e03fc5 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:33.931706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1055/2067] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:00:33.936236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4722dfd-0cfd-4d3d-aaa3-4099870a74dd None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:33.946996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4722dfd-0cfd-4d3d-aaa3-4099870a74dd None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:33.958455 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109ed29-3d66-4892-80c0-d8721ebad4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:33.959922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109ed29-3d66-4892-80c0-d8721ebad4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:33.960382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109ed29-3d66-4892-80c0-d8721ebad4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:33.960960 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109ed29-3d66-4892-80c0-d8721ebad4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:33.961758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109ed29-3d66-4892-80c0-d8721ebad4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:33.964386 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109ed29-3d66-4892-80c0-d8721ebad4b5 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:33.998316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1013/2068] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:00:33 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:00:34.003549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde8bc7-d30c-4663-830e-32c860e03fc5 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:34.014162 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde8bc7-d30c-4663-830e-32c860e03fc5 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:34.063462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:34.080289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6989ae-5675-4d9d-b48c-97482aab105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:34.080992 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6989ae-5675-4d9d-b48c-97482aab105d admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:34.083067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6989ae-5675-4d9d-b48c-97482aab105d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:34.083543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6989ae-5675-4d9d-b48c-97482aab105d admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:34.084363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6989ae-5675-4d9d-b48c-97482aab105d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:34.086307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6989ae-5675-4d9d-b48c-97482aab105d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:34.113628 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1056/2069] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:00:34 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:00:34.118205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109ed29-3d66-4892-80c0-d8721ebad4b5 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:34.129676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109ed29-3d66-4892-80c0-d8721ebad4b5 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:34.135446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b47a5-c301-4c74-ad16-348ab8cfc921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:34.136293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b47a5-c301-4c74-ad16-348ab8cfc921 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:34.136590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b47a5-c301-4c74-ad16-348ab8cfc921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:34.136943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b47a5-c301-4c74-ad16-348ab8cfc921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:34.137699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b47a5-c301-4c74-ad16-348ab8cfc921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:34.139976 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b47a5-c301-4c74-ad16-348ab8cfc921 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:34.154107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1014/2070] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:00:34.156763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6989ae-5675-4d9d-b48c-97482aab105d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:34.167307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6989ae-5675-4d9d-b48c-97482aab105d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:34.173624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97fa99-ee83-44b5-aeab-8ba9d61a1e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:34.174136 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97fa99-ee83-44b5-aeab-8ba9d61a1e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:34.174422 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97fa99-ee83-44b5-aeab-8ba9d61a1e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:34.174695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97fa99-ee83-44b5-aeab-8ba9d61a1e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:34.175914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97fa99-ee83-44b5-aeab-8ba9d61a1e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:34.177014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97fa99-ee83-44b5-aeab-8ba9d61a1e5d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:34.191077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1057/2071] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 18:00:34.195458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b47a5-c301-4c74-ad16-348ab8cfc921 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:34.205983 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b47a5-c301-4c74-ad16-348ab8cfc921 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:34.210705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035db5b-2652-487c-b92c-d85bd8c11b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:34.211366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035db5b-2652-487c-b92c-d85bd8c11b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:34.211648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035db5b-2652-487c-b92c-d85bd8c11b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:34.211963 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035db5b-2652-487c-b92c-d85bd8c11b9b admin admin] PATH_INFO: `/v3/projects/16610892c7d34b28a2ee5313a4bea23b/users/74943fd4c96b43e8a32989e7ed74cdf2/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:34.233797 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035db5b-2652-487c-b92c-d85bd8c11b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16610892c7d34b28a2ee5313a4bea23b, user_id=74943fd4c96b43e8a32989e7ed74cdf2, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:34.236537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035db5b-2652-487c-b92c-d85bd8c11b9b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:34.252350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1015/2072] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:00:34 2022] PUT /identity/v3/projects/16610892c7d34b28a2ee5313a4bea23b/users/74943fd4c96b43e8a32989e7ed74cdf2/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:00:34.257617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5017c-5558-4526-a373-2bf08f672b9c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:34.257898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5017c-5558-4526-a373-2bf08f672b9c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:34.258198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5017c-5558-4526-a373-2bf08f672b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:34.277648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e5c5017c-5558-4526-a373-2bf08f672b9c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:34.278839 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e5c5017c-5558-4526-a373-2bf08f672b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:34.342090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1058/2073] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:34.362572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035db5b-2652-487c-b92c-d85bd8c11b9b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:34.364519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a035db5b-2652-487c-b92c-d85bd8c11b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:34.442763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035db5b-2652-487c-b92c-d85bd8c11b9b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:34.446944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fae86-9034-4710-a753-85ef0c43f616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:34.447818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fae86-9034-4710-a753-85ef0c43f616 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:34.448195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fae86-9034-4710-a753-85ef0c43f616 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:34.448543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fae86-9034-4710-a753-85ef0c43f616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:34.457065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:34.461997 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fae86-9034-4710-a753-85ef0c43f616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:34.464934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fae86-9034-4710-a753-85ef0c43f616 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:34.477443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1016/2074] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:00:37.169156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5017c-5558-4526-a373-2bf08f672b9c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:37.171542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e5c5017c-5558-4526-a373-2bf08f672b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:37.191129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5017c-5558-4526-a373-2bf08f672b9c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:37.195847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd1ead-8135-4f4e-85f0-72525a0f699a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:37.196611 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd1ead-8135-4f4e-85f0-72525a0f699a service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:37.197052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd1ead-8135-4f4e-85f0-72525a0f699a service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:37.197524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd1ead-8135-4f4e-85f0-72525a0f699a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:37.199742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5fbd1ead-8135-4f4e-85f0-72525a0f699a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:37.296069 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd1ead-8135-4f4e-85f0-72525a0f699a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:37.297922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd1ead-8135-4f4e-85f0-72525a0f699a service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:37.309843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1059/2075] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:39.547219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9f5c4-61a2-493e-b4a7-91219b4d6b7e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:39.547692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9f5c4-61a2-493e-b4a7-91219b4d6b7e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:39.548321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9f5c4-61a2-493e-b4a7-91219b4d6b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:39.580236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f2e9f5c4-61a2-493e-b4a7-91219b4d6b7e None None] MFA Rules not processed for user `74943fd4c96b43e8a32989e7ed74cdf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:39.582288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f2e9f5c4-61a2-493e-b4a7-91219b4d6b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:39.670216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1017/2076] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:39.671727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd1ead-8135-4f4e-85f0-72525a0f699a service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:39.684428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd1ead-8135-4f4e-85f0-72525a0f699a service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:39.690246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d826b-be77-4dc8-bbf8-2128c136493b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:39.690978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d826b-be77-4dc8-bbf8-2128c136493b admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:39.691348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d826b-be77-4dc8-bbf8-2128c136493b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:39.691704 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d826b-be77-4dc8-bbf8-2128c136493b admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:39.692473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d826b-be77-4dc8-bbf8-2128c136493b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:39.694498 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d826b-be77-4dc8-bbf8-2128c136493b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:39.726047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1060/2077] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:00:39 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:00:39.729930 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9f5c4-61a2-493e-b4a7-91219b4d6b7e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:39.742627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9f5c4-61a2-493e-b4a7-91219b4d6b7e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:39.807314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85307b36-d0a7-4f4d-973c-b43de81ab28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:39.807970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85307b36-d0a7-4f4d-973c-b43de81ab28c admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:39.808242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85307b36-d0a7-4f4d-973c-b43de81ab28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:39.808543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85307b36-d0a7-4f4d-973c-b43de81ab28c admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:39.809185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85307b36-d0a7-4f4d-973c-b43de81ab28c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:39.811130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85307b36-d0a7-4f4d-973c-b43de81ab28c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:39.835293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1018/2078] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:00:39 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:00:39.839215 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d826b-be77-4dc8-bbf8-2128c136493b admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:39.853818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d826b-be77-4dc8-bbf8-2128c136493b admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:39.859199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b3351-50d6-4927-9743-799d31a8d37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:39.859964 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b3351-50d6-4927-9743-799d31a8d37d admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:39.860237 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b3351-50d6-4927-9743-799d31a8d37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:39.860563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b3351-50d6-4927-9743-799d31a8d37d admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:39.861296 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b3351-50d6-4927-9743-799d31a8d37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:39.862960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b3351-50d6-4927-9743-799d31a8d37d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:39.877066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1061/2079] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:00:39.887098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85307b36-d0a7-4f4d-973c-b43de81ab28c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:39.898259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85307b36-d0a7-4f4d-973c-b43de81ab28c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:39.933952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c46bcf-d420-4edf-82d3-15beb052d19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:39.934642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c46bcf-d420-4edf-82d3-15beb052d19c admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:39.934925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c46bcf-d420-4edf-82d3-15beb052d19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:39.935201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c46bcf-d420-4edf-82d3-15beb052d19c admin admin] PATH_INFO: `/v3/projects/90bee264213b45d9a287167aa9d7475e/users/5bad6b84ac1141618a82f995b3422ddf/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:39.953136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c46bcf-d420-4edf-82d3-15beb052d19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=90bee264213b45d9a287167aa9d7475e, user_id=5bad6b84ac1141618a82f995b3422ddf, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:39.955801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c46bcf-d420-4edf-82d3-15beb052d19c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:39.982840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1019/2080] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:00:39 2022] PUT /identity/v3/projects/90bee264213b45d9a287167aa9d7475e/users/5bad6b84ac1141618a82f995b3422ddf/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:00:39.987067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b3351-50d6-4927-9743-799d31a8d37d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:39.997452 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b3351-50d6-4927-9743-799d31a8d37d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:40.028811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:40.082381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a6e06-93df-45b9-bd91-01066ce99e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:40.083002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a6e06-93df-45b9-bd91-01066ce99e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:40.083277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a6e06-93df-45b9-bd91-01066ce99e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:40.083692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a6e06-93df-45b9-bd91-01066ce99e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:40.084474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a6e06-93df-45b9-bd91-01066ce99e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:40.086094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a6e06-93df-45b9-bd91-01066ce99e76 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:40.099629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1062/2081] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:00:40.103782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c46bcf-d420-4edf-82d3-15beb052d19c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:40.116522 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c46bcf-d420-4edf-82d3-15beb052d19c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:40.121658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812144d-73d6-4559-a3a7-3ac8771881b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:40.125682 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812144d-73d6-4559-a3a7-3ac8771881b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:40.125682 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812144d-73d6-4559-a3a7-3ac8771881b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:40.125682 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812144d-73d6-4559-a3a7-3ac8771881b1 admin admin] PATH_INFO: `/v3/projects/90bee264213b45d9a287167aa9d7475e/users/5bad6b84ac1141618a82f995b3422ddf/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:40.127240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812144d-73d6-4559-a3a7-3ac8771881b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90bee264213b45d9a287167aa9d7475e, user_id=5bad6b84ac1141618a82f995b3422ddf, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:40.130018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812144d-73d6-4559-a3a7-3ac8771881b1 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:40.142774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1020/2082] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:00:40 2022] PUT /identity/v3/projects/90bee264213b45d9a287167aa9d7475e/users/5bad6b84ac1141618a82f995b3422ddf/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:00:43.181866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a6e06-93df-45b9-bd91-01066ce99e76 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:43.277357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a6e06-93df-45b9-bd91-01066ce99e76 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:43.353799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5b67b-589a-492e-a7ed-d9c69eec2afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:43.354682 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5b67b-589a-492e-a7ed-d9c69eec2afa service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:43.355063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5b67b-589a-492e-a7ed-d9c69eec2afa service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:43.355443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5b67b-589a-492e-a7ed-d9c69eec2afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:43.357405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b3d5b67b-589a-492e-a7ed-d9c69eec2afa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:43.369288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5b67b-589a-492e-a7ed-d9c69eec2afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:43.370873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5b67b-589a-492e-a7ed-d9c69eec2afa service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:43.382336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1063/2083] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:44.930476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812144d-73d6-4559-a3a7-3ac8771881b1 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:44.932923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b812144d-73d6-4559-a3a7-3ac8771881b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:45.020452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:45.083712 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812144d-73d6-4559-a3a7-3ac8771881b1 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:45.090513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7ddc8-1444-4a3b-b629-fdedb0111738 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:45.091634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7ddc8-1444-4a3b-b629-fdedb0111738 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:45.092112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7ddc8-1444-4a3b-b629-fdedb0111738 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:45.092952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7ddc8-1444-4a3b-b629-fdedb0111738 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:45.095199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-bfa7ddc8-1444-4a3b-b629-fdedb0111738 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:45.214542 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7ddc8-1444-4a3b-b629-fdedb0111738 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:45.216616 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7ddc8-1444-4a3b-b629-fdedb0111738 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:45.232073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1021/2084] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:46.120638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984d58c-d9f9-418e-9d64-72bb98d5963c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:46.121533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984d58c-d9f9-418e-9d64-72bb98d5963c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:46.122189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984d58c-d9f9-418e-9d64-72bb98d5963c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:46.161675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-4984d58c-d9f9-418e-9d64-72bb98d5963c None None] MFA Rules not processed for user `5bad6b84ac1141618a82f995b3422ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:46.162891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4984d58c-d9f9-418e-9d64-72bb98d5963c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:46.232258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1064/2085] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:46.246887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7ddc8-1444-4a3b-b629-fdedb0111738 service nova] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:46.248513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-bfa7ddc8-1444-4a3b-b629-fdedb0111738 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:46.265651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7ddc8-1444-4a3b-b629-fdedb0111738 service nova] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:46.271224 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e48ed6-e0e0-430e-bfed-9d1fff4df155 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:46.272099 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e48ed6-e0e0-430e-bfed-9d1fff4df155 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:46.272461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e48ed6-e0e0-430e-bfed-9d1fff4df155 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:46.272655 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e48ed6-e0e0-430e-bfed-9d1fff4df155 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:46.338520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e48ed6-e0e0-430e-bfed-9d1fff4df155 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:46.340146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e48ed6-e0e0-430e-bfed-9d1fff4df155 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:46.350924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1022/2086] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:48.179075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae85706-1444-47c6-bee2-1bb3771a9bea None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:48.179899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae85706-1444-47c6-bee2-1bb3771a9bea None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:48.180320 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae85706-1444-47c6-bee2-1bb3771a9bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:48.252974 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7ae85706-1444-47c6-bee2-1bb3771a9bea None None] MFA Rules not processed for user `74943fd4c96b43e8a32989e7ed74cdf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:48.254251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7ae85706-1444-47c6-bee2-1bb3771a9bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:48.276695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1065/2087] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:00:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:48.276971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e48ed6-e0e0-430e-bfed-9d1fff4df155 service nova] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:48.292223 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e48ed6-e0e0-430e-bfed-9d1fff4df155 service nova] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:48.305283 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c0f20-f4d0-4d65-aef4-6f8006bf549a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:48.305863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c0f20-f4d0-4d65-aef4-6f8006bf549a service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:48.306140 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c0f20-f4d0-4d65-aef4-6f8006bf549a service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:48.306411 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c0f20-f4d0-4d65-aef4-6f8006bf549a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:48.313242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-694c0f20-f4d0-4d65-aef4-6f8006bf549a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:48.354163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c0f20-f4d0-4d65-aef4-6f8006bf549a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:48.355776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c0f20-f4d0-4d65-aef4-6f8006bf549a service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:48.387149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1023/2088] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:00:54.452765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86043f3c-975b-4d61-b18f-61958e55376b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:54.455024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86043f3c-975b-4d61-b18f-61958e55376b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:54.455478 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86043f3c-975b-4d61-b18f-61958e55376b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:54.470705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:54.485857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-86043f3c-975b-4d61-b18f-61958e55376b None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:00:54.487979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-86043f3c-975b-4d61-b18f-61958e55376b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:54.556197 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1066/2089] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 18:00:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:00:54.567091 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c0f20-f4d0-4d65-aef4-6f8006bf549a service nova] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:54.568560 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-694c0f20-f4d0-4d65-aef4-6f8006bf549a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:54.641471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:00:54.648431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c0f20-f4d0-4d65-aef4-6f8006bf549a service nova] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:54.653597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be262d-a19a-42ff-beba-427b1a0948c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:54.654213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be262d-a19a-42ff-beba-427b1a0948c0 service swift] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:54.654494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be262d-a19a-42ff-beba-427b1a0948c0 service swift] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:54.654762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be262d-a19a-42ff-beba-427b1a0948c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:54.663862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be262d-a19a-42ff-beba-427b1a0948c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:54.665467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be262d-a19a-42ff-beba-427b1a0948c0 service swift] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:54.674623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1024/2090] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 18:00:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 18:00:54.720756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86043f3c-975b-4d61-b18f-61958e55376b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:00:54.740499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86043f3c-975b-4d61-b18f-61958e55376b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:00:54.753130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04549c0f-94c8-44a4-aec8-4475a81ba9dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:00:54.753813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04549c0f-94c8-44a4-aec8-4475a81ba9dd service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:00:54.754096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04549c0f-94c8-44a4-aec8-4475a81ba9dd service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:00:54.754363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04549c0f-94c8-44a4-aec8-4475a81ba9dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:00:54.760909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-04549c0f-94c8-44a4-aec8-4475a81ba9dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:00:54.777695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04549c0f-94c8-44a4-aec8-4475a81ba9dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:00:54.780168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04549c0f-94c8-44a4-aec8-4475a81ba9dd service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:00:54.809647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1067/2091] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:00:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:04.670359 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:04.792353 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=30 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:10.032065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be262d-a19a-42ff-beba-427b1a0948c0 service swift] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:10.041257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be262d-a19a-42ff-beba-427b1a0948c0 service swift] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:10.044837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb10ef3-275a-4794-ad53-c85daa94dffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:10.045418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb10ef3-275a-4794-ad53-c85daa94dffb service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:10.045693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb10ef3-275a-4794-ad53-c85daa94dffb service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:10.045964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb10ef3-275a-4794-ad53-c85daa94dffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:10.048536 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0eb10ef3-275a-4794-ad53-c85daa94dffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:10.057834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb10ef3-275a-4794-ad53-c85daa94dffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:10.059240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb10ef3-275a-4794-ad53-c85daa94dffb service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:10.067846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1025/2092] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:10.189497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609ef22-8191-4644-8ed0-93963de51d8f None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:10.190051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609ef22-8191-4644-8ed0-93963de51d8f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:10.190051 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609ef22-8191-4644-8ed0-93963de51d8f None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:10.191253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1068/2093] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 18:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 18:01:10.197152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b2097-3751-4d0b-8302-296d92517dbc None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:10.197528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b2097-3751-4d0b-8302-296d92517dbc None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:10.197970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b2097-3751-4d0b-8302-296d92517dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:10.218821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-4f4b2097-3751-4d0b-8302-296d92517dbc None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:10.220259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-4f4b2097-3751-4d0b-8302-296d92517dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:10.229592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1026/2094] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 18:01:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:01:10.241924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609ef22-8191-4644-8ed0-93963de51d8f None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:10.243726 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-4609ef22-8191-4644-8ed0-93963de51d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:10.311929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609ef22-8191-4644-8ed0-93963de51d8f None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:10.317491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ae1a4-ae4b-487d-8024-83d121d821b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3566d85c9c4c559ba672269ab38f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:10.318366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ae1a4-ae4b-487d-8024-83d121d821b8 service glance] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:10.318642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ae1a4-ae4b-487d-8024-83d121d821b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:10.318906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ae1a4-ae4b-487d-8024-83d121d821b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:10.337337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ae1a4-ae4b-487d-8024-83d121d821b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:10.339027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ae1a4-ae4b-487d-8024-83d121d821b8 service glance] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:10.359527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1069/2095] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:10.441040 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98825090-1e68-4c60-a1d6-036b9cb0ef30 None None] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:10.441572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98825090-1e68-4c60-a1d6-036b9cb0ef30 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:10.441920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98825090-1e68-4c60-a1d6-036b9cb0ef30 None None] PATH_INFO: `/` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:10.443325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1027/2096] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 18:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 18:01:10.460403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60c2ec-f3a1-4c0d-95b7-b327ad904368 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:10.460403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60c2ec-f3a1-4c0d-95b7-b327ad904368 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:10.460403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60c2ec-f3a1-4c0d-95b7-b327ad904368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:10.470875 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1a60c2ec-f3a1-4c0d-95b7-b327ad904368 None None] MFA Rules not processed for user `3c17f274e4c14f2f99650844642437a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:10.472127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1a60c2ec-f3a1-4c0d-95b7-b327ad904368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:10.478600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1070/2097] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 18:01:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:10.490418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98825090-1e68-4c60-a1d6-036b9cb0ef30 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:10.501934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98825090-1e68-4c60-a1d6-036b9cb0ef30 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:10.505451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbacf05-903c-4d02-bb2e-c0d6b4ae48ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:10.506036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbacf05-903c-4d02-bb2e-c0d6b4ae48ff service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:10.506311 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbacf05-903c-4d02-bb2e-c0d6b4ae48ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:10.506581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbacf05-903c-4d02-bb2e-c0d6b4ae48ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:10.513505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbacf05-903c-4d02-bb2e-c0d6b4ae48ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:10.514940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbacf05-903c-4d02-bb2e-c0d6b4ae48ff service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:10.523137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1028/2098] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:10.604122 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d88211-2e9a-4425-b752-d2b07316f1de None None] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:10.604400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d88211-2e9a-4425-b752-d2b07316f1de None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:10.604666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d88211-2e9a-4425-b752-d2b07316f1de None None] PATH_INFO: `/` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:10.606019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1071/2099] 10.222.0.27 () {54 vars in 921 bytes} [Fri Oct 14 18:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 18:01:10.610881 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959402ce-507a-4612-90b8-6ab853436946 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:10.611167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959402ce-507a-4612-90b8-6ab853436946 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:10.611440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959402ce-507a-4612-90b8-6ab853436946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:10.633564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-959402ce-507a-4612-90b8-6ab853436946 None None] MFA Rules not processed for user `e42968016de8470c933655e921fb3394`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:10.634789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-959402ce-507a-4612-90b8-6ab853436946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:10.646359 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1029/2100] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Oct 14 18:01:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:10.658384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d88211-2e9a-4425-b752-d2b07316f1de None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:10.671266 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d88211-2e9a-4425-b752-d2b07316f1de None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:10.676786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27fb45-a1b1-47f7-94b2-5fda601182dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:10.677463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27fb45-a1b1-47f7-94b2-5fda601182dc service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:10.677746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27fb45-a1b1-47f7-94b2-5fda601182dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:10.678881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27fb45-a1b1-47f7-94b2-5fda601182dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:10.693862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27fb45-a1b1-47f7-94b2-5fda601182dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:10.695432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27fb45-a1b1-47f7-94b2-5fda601182dc service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:10.720943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1072/2101] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:12.692173 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959402ce-507a-4612-90b8-6ab853436946 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:12.709099 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959402ce-507a-4612-90b8-6ab853436946 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:12.727412 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f2a2-8d01-45da-bedc-75633fec14a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:12.728065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f2a2-8d01-45da-bedc-75633fec14a2 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:12.728354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f2a2-8d01-45da-bedc-75633fec14a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:12.728628 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f2a2-8d01-45da-bedc-75633fec14a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:12.731589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f475f2a2-8d01-45da-bedc-75633fec14a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:12.746478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475f2a2-8d01-45da-bedc-75633fec14a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:12.748462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475f2a2-8d01-45da-bedc-75633fec14a2 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:12.779062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1030/2102] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:13.800366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5155-0137-40d3-b7a1-7b50f35c79f3 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:13.800850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5155-0137-40d3-b7a1-7b50f35c79f3 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:13.800932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5155-0137-40d3-b7a1-7b50f35c79f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:13.820552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1d7d5155-0137-40d3-b7a1-7b50f35c79f3 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:13.821671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1d7d5155-0137-40d3-b7a1-7b50f35c79f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:13.828918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1073/2103] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 18:01:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:13.841339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f2a2-8d01-45da-bedc-75633fec14a2 service nova] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:13.852555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f2a2-8d01-45da-bedc-75633fec14a2 service nova] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:13.857426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dc262-62f1-4f40-a118-720ccd3d2232 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:13.858021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dc262-62f1-4f40-a118-720ccd3d2232 service swift] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:13.858298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dc262-62f1-4f40-a118-720ccd3d2232 service swift] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:13.858565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dc262-62f1-4f40-a118-720ccd3d2232 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:13.867707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089dc262-62f1-4f40-a118-720ccd3d2232 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:13.869301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089dc262-62f1-4f40-a118-720ccd3d2232 service swift] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:13.878527 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1031/2104] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 18:01:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 18:01:18.802651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d5155-0137-40d3-b7a1-7b50f35c79f3 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:18.812227 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d5155-0137-40d3-b7a1-7b50f35c79f3 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:18.816754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aef8d5-7f05-4400-95c0-e5aacbe67a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:18.817340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aef8d5-7f05-4400-95c0-e5aacbe67a9d service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:18.817608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aef8d5-7f05-4400-95c0-e5aacbe67a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:18.817873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aef8d5-7f05-4400-95c0-e5aacbe67a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:18.819624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e0aef8d5-7f05-4400-95c0-e5aacbe67a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:18.893357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aef8d5-7f05-4400-95c0-e5aacbe67a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:18.894921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aef8d5-7f05-4400-95c0-e5aacbe67a9d service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:18.908673 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1074/2105] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:23.147652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dc262-62f1-4f40-a118-720ccd3d2232 service swift] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:23.149186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-089dc262-62f1-4f40-a118-720ccd3d2232 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:23.171439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dc262-62f1-4f40-a118-720ccd3d2232 service swift] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:23.177043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:23.177665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:23.177940 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:23.178212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] PATH_INFO: `/v3/users/762e1e38e4234803991187db9b870b9c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:23.179376 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=762e1e38e4234803991187db9b870b9c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:23.181334 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:23.226734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762e1e38e4234803991187db9b870b9c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:23.227389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762e1e38e4234803991187db9b870b9c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:23.234304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762e1e38e4234803991187db9b870b9c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:23.235484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762e1e38e4234803991187db9b870b9c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:23.247366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9519ee06-ad97-4622-a027-3dd8b09d2b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762e1e38e4234803991187db9b870b9c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:23.263294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1032/2106] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:01:23 2022] DELETE /identity/v3/users/762e1e38e4234803991187db9b870b9c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:23.439748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aef8d5-7f05-4400-95c0-e5aacbe67a9d service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:23.450607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aef8d5-7f05-4400-95c0-e5aacbe67a9d service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:23.507950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:23.508572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:23.508846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:23.509133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] PATH_INFO: `/v3/projects/f5488304972c48fab9a2b9711cf7e9c8` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:23.510217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5488304972c48fab9a2b9711cf7e9c8)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:23.512160 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:23.524015 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:23.559654 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5488304972c48fab9a2b9711cf7e9c8', 'request_id': 'req-b28617a7-7233-460d-ab4d-dc9d42c535e2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:23.565806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5488304972c48fab9a2b9711cf7e9c8', 'request_id': 'req-b28617a7-7233-460d-ab4d-dc9d42c535e2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:23.572777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5488304972c48fab9a2b9711cf7e9c8', 'request_id': 'req-b28617a7-7233-460d-ab4d-dc9d42c535e2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:23.585478 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] The token cache is being invalidate because project f5488304972c48fab9a2b9711cf7e9c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:23.585807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:23.586657 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b28617a7-7233-460d-ab4d-dc9d42c535e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:23.588583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1075/2107] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:23 2022] DELETE /identity/v3/projects/f5488304972c48fab9a2b9711cf7e9c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:01:23.593995 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2726c29-41e7-44f6-a92e-1f2d81e3473d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:23.594306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2726c29-41e7-44f6-a92e-1f2d81e3473d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:23.594635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2726c29-41e7-44f6-a92e-1f2d81e3473d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:23.614448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-d2726c29-41e7-44f6-a92e-1f2d81e3473d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:23.615570 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d2726c29-41e7-44f6-a92e-1f2d81e3473d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:23.675214 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1033/2108] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:23.682907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08036a76-5adf-4998-8589-1caa9b0c76b8 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:23.683182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08036a76-5adf-4998-8589-1caa9b0c76b8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:23.683473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08036a76-5adf-4998-8589-1caa9b0c76b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:23.704426 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-08036a76-5adf-4998-8589-1caa9b0c76b8 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:23.705540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-08036a76-5adf-4998-8589-1caa9b0c76b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:23.713386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1076/2109] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:23.717421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2726c29-41e7-44f6-a92e-1f2d81e3473d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:23.727017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2726c29-41e7-44f6-a92e-1f2d81e3473d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:23.731559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcaa97-5937-404e-b260-30fe955e1224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:23.732193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcaa97-5937-404e-b260-30fe955e1224 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:23.732470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcaa97-5937-404e-b260-30fe955e1224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:23.732740 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcaa97-5937-404e-b260-30fe955e1224 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:23.733374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcaa97-5937-404e-b260-30fe955e1224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:01:23.733671 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcaa97-5937-404e-b260-30fe955e1224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:23.735253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcaa97-5937-404e-b260-30fe955e1224 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:23.736705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:23.748381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1034/2110] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:01:23.753109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08036a76-5adf-4998-8589-1caa9b0c76b8 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:23.762247 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08036a76-5adf-4998-8589-1caa9b0c76b8 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:23.767501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f1190-5010-470b-bad9-8b65b5511d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:23.768114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f1190-5010-470b-bad9-8b65b5511d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:23.768390 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f1190-5010-470b-bad9-8b65b5511d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:23.768659 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f1190-5010-470b-bad9-8b65b5511d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:23.769288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f1190-5010-470b-bad9-8b65b5511d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:23.771181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f1190-5010-470b-bad9-8b65b5511d1a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:23.798013 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1077/2111] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:23 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:01:23.802163 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcaa97-5937-404e-b260-30fe955e1224 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:23.812086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcaa97-5937-404e-b260-30fe955e1224 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:23.868391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb833ac2-c05c-4042-9c03-51070e238f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:23.869025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb833ac2-c05c-4042-9c03-51070e238f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:23.869303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb833ac2-c05c-4042-9c03-51070e238f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:23.869573 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb833ac2-c05c-4042-9c03-51070e238f1a admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:23.870198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb833ac2-c05c-4042-9c03-51070e238f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:23.872142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb833ac2-c05c-4042-9c03-51070e238f1a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:23.894134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1035/2112] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:01:23 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:23.898180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f1190-5010-470b-bad9-8b65b5511d1a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:23.908622 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f1190-5010-470b-bad9-8b65b5511d1a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:23.913286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592ea5b-4f21-4c06-b427-dbf376123b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:23.913877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592ea5b-4f21-4c06-b427-dbf376123b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:23.914151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592ea5b-4f21-4c06-b427-dbf376123b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:23.914421 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592ea5b-4f21-4c06-b427-dbf376123b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:23.915083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592ea5b-4f21-4c06-b427-dbf376123b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:23.916718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592ea5b-4f21-4c06-b427-dbf376123b9a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:23.926968 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1078/2113] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:23.930853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb833ac2-c05c-4042-9c03-51070e238f1a admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:23.941658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb833ac2-c05c-4042-9c03-51070e238f1a admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:23.946636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aea999-3f60-4b15-825e-002f77269628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:23.947234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aea999-3f60-4b15-825e-002f77269628 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:23.947512 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aea999-3f60-4b15-825e-002f77269628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:23.947788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aea999-3f60-4b15-825e-002f77269628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:23.948476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aea999-3f60-4b15-825e-002f77269628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:23.950066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aea999-3f60-4b15-825e-002f77269628 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:23.960982 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1036/2114] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:23.965141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592ea5b-4f21-4c06-b427-dbf376123b9a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:23.975471 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592ea5b-4f21-4c06-b427-dbf376123b9a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:23.980209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a96e1-acc0-4f39-bc55-dd4c05b0b792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:23.980855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a96e1-acc0-4f39-bc55-dd4c05b0b792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:23.981145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a96e1-acc0-4f39-bc55-dd4c05b0b792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:23.981394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a96e1-acc0-4f39-bc55-dd4c05b0b792 admin admin] PATH_INFO: `/v3/projects/d559f9074d014d0aa22b1346dc439bc5/users/0550e66e44eb4e4e80b3513341087faa/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:23.997645 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a96e1-acc0-4f39-bc55-dd4c05b0b792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d559f9074d014d0aa22b1346dc439bc5, user_id=0550e66e44eb4e4e80b3513341087faa, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:24.000388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a96e1-acc0-4f39-bc55-dd4c05b0b792 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:24.016470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1079/2115] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:23 2022] PUT /identity/v3/projects/d559f9074d014d0aa22b1346dc439bc5/users/0550e66e44eb4e4e80b3513341087faa/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:01:24.028408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7830c7b-ca43-4e5f-bad2-8e668bca25d9 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:24.028689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7830c7b-ca43-4e5f-bad2-8e668bca25d9 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:24.028962 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7830c7b-ca43-4e5f-bad2-8e668bca25d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:24.057965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-e7830c7b-ca43-4e5f-bad2-8e668bca25d9 None None] MFA Rules not processed for user `0550e66e44eb4e4e80b3513341087faa`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:24.059201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e7830c7b-ca43-4e5f-bad2-8e668bca25d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:24.126758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1037/2116] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:24.138161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a96e1-acc0-4f39-bc55-dd4c05b0b792 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:24.140480 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3c9a96e1-acc0-4f39-bc55-dd4c05b0b792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:24.215719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a96e1-acc0-4f39-bc55-dd4c05b0b792 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:24.221730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a52010-8867-4b31-9c76-338891b31ec6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:24.222327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a52010-8867-4b31-9c76-338891b31ec6 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:24.222600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a52010-8867-4b31-9c76-338891b31ec6 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:24.222865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a52010-8867-4b31-9c76-338891b31ec6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:24.230671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a52010-8867-4b31-9c76-338891b31ec6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:24.232287 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a52010-8867-4b31-9c76-338891b31ec6 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:24.241948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1080/2117] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:01:31.521637 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7830c7b-ca43-4e5f-bad2-8e668bca25d9 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:31.585737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7830c7b-ca43-4e5f-bad2-8e668bca25d9 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:31.590011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:31.590641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:31.591537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:31.591807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] PATH_INFO: `/v3/users/0550e66e44eb4e4e80b3513341087faa` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:31.592946 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0550e66e44eb4e4e80b3513341087faa)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:31.594842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:31.607300 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:31.620875 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a52010-8867-4b31-9c76-338891b31ec6 service nova] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:31.622674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-96a52010-8867-4b31-9c76-338891b31ec6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:31.648761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0550e66e44eb4e4e80b3513341087faa'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:31.649417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0550e66e44eb4e4e80b3513341087faa'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:31.653425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:31.657207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0550e66e44eb4e4e80b3513341087faa'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:31.658632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0550e66e44eb4e4e80b3513341087faa'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:31.672322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6a9ce348-33d8-4538-bb8e-038575d62669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0550e66e44eb4e4e80b3513341087faa'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:31.687209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a52010-8867-4b31-9c76-338891b31ec6 service nova] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:31.687680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1038/2118] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:01:31 2022] DELETE /identity/v3/users/0550e66e44eb4e4e80b3513341087faa => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:31.692714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11795c-21f5-4aa6-896a-37b7b79f8fbe None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:31.692995 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11795c-21f5-4aa6-896a-37b7b79f8fbe None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:31.693268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11795c-21f5-4aa6-896a-37b7b79f8fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:31.713650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-aa11795c-21f5-4aa6-896a-37b7b79f8fbe None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:31.714842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-aa11795c-21f5-4aa6-896a-37b7b79f8fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:31.738811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53cc9b-b867-443e-bf21-b5b290e221c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:31.739412 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53cc9b-b867-443e-bf21-b5b290e221c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:31.739684 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53cc9b-b867-443e-bf21-b5b290e221c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:31.739979 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53cc9b-b867-443e-bf21-b5b290e221c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:31.741686 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cf53cc9b-b867-443e-bf21-b5b290e221c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:31.777973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1039/2119] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:01:31.788197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11795c-21f5-4aa6-896a-37b7b79f8fbe None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:31.789553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-aa11795c-21f5-4aa6-896a-37b7b79f8fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:31.802572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11795c-21f5-4aa6-896a-37b7b79f8fbe None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:31.805198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53cc9b-b867-443e-bf21-b5b290e221c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:31.806432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17411c0b-1293-489d-8af5-6c72b15afe1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:31.806670 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53cc9b-b867-443e-bf21-b5b290e221c8 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:31.807032 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17411c0b-1293-489d-8af5-6c72b15afe1e service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:31.807306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17411c0b-1293-489d-8af5-6c72b15afe1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:31.807571 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17411c0b-1293-489d-8af5-6c72b15afe1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:31.816930 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17411c0b-1293-489d-8af5-6c72b15afe1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:31.817075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1081/2120] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:01:31.818521 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17411c0b-1293-489d-8af5-6c72b15afe1e service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:31.828496 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1040/2121] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:01:32.263156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53cc9b-b867-443e-bf21-b5b290e221c8 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:32.265003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cf53cc9b-b867-443e-bf21-b5b290e221c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:32.285132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53cc9b-b867-443e-bf21-b5b290e221c8 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:32.290616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:32.291322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.291661 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.292018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] PATH_INFO: `/v3/projects/d559f9074d014d0aa22b1346dc439bc5` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.293235 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d559f9074d014d0aa22b1346dc439bc5)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:32.295203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:32.357006 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd559f9074d014d0aa22b1346dc439bc5', 'request_id': 'req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:32.363555 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd559f9074d014d0aa22b1346dc439bc5', 'request_id': 'req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:32.372023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd559f9074d014d0aa22b1346dc439bc5', 'request_id': 'req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:32.385300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] The token cache is being invalidate because project d559f9074d014d0aa22b1346dc439bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:32.385780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:32.386285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-ecfaa0c1-1a8a-4187-875b-bf47664b8fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:32.393618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1082/2122] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:32 2022] DELETE /identity/v3/projects/d559f9074d014d0aa22b1346dc439bc5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:32.398631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d684f2-3a3d-43a7-888d-63f3cc184574 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.398988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d684f2-3a3d-43a7-888d-63f3cc184574 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.399348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d684f2-3a3d-43a7-888d-63f3cc184574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.421497 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-70d684f2-3a3d-43a7-888d-63f3cc184574 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:32.423384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-70d684f2-3a3d-43a7-888d-63f3cc184574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:32.490391 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1041/2123] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:01:32.498674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118b48b-4d96-46da-bd21-41cc618023e1 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.499194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118b48b-4d96-46da-bd21-41cc618023e1 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.499756 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118b48b-4d96-46da-bd21-41cc618023e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.530180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f536-137b-4f2c-a218-5302c986bda1 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.530974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f536-137b-4f2c-a218-5302c986bda1 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.531412 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f536-137b-4f2c-a218-5302c986bda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.546080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f118b48b-4d96-46da-bd21-41cc618023e1 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:32.547360 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f118b48b-4d96-46da-bd21-41cc618023e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:32.553932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ae74f536-137b-4f2c-a218-5302c986bda1 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:32.555426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ae74f536-137b-4f2c-a218-5302c986bda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:32.557881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1083/2124] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:32.561770 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118b48b-4d96-46da-bd21-41cc618023e1 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:32.563801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:32.569105 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:32.573448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118b48b-4d96-46da-bd21-41cc618023e1 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:32.584625 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c02e28-569c-4ecb-871e-c2ed992ce888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:32.585520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c02e28-569c-4ecb-871e-c2ed992ce888 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.585869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c02e28-569c-4ecb-871e-c2ed992ce888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.586275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c02e28-569c-4ecb-871e-c2ed992ce888 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.587052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c02e28-569c-4ecb-871e-c2ed992ce888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:01:32.587416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c02e28-569c-4ecb-871e-c2ed992ce888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:32.589119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c02e28-569c-4ecb-871e-c2ed992ce888 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:32.604694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1084/2125] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 18:01:32.609009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c02e28-569c-4ecb-871e-c2ed992ce888 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:32.620537 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c02e28-569c-4ecb-871e-c2ed992ce888 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:32.626636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097f1c2-34b2-41d3-86de-b094c356f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:32.627347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097f1c2-34b2-41d3-86de-b094c356f265 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.627723 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097f1c2-34b2-41d3-86de-b094c356f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.628122 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097f1c2-34b2-41d3-86de-b094c356f265 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.628891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097f1c2-34b2-41d3-86de-b094c356f265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:32.631115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097f1c2-34b2-41d3-86de-b094c356f265 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:32.632355 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1042/2126] 10.222.0.27 () {58 vars in 967 bytes} [Fri Oct 14 18:01:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:01:32.657323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74f536-137b-4f2c-a218-5302c986bda1 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:32.659640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ae74f536-137b-4f2c-a218-5302c986bda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:32.690106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1085/2127] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:32 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:01:32.694721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097f1c2-34b2-41d3-86de-b094c356f265 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:32.747725 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74f536-137b-4f2c-a218-5302c986bda1 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:32.777455 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097f1c2-34b2-41d3-86de-b094c356f265 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:32.783765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c3f07-eca6-41af-a2a9-b48626dcf306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:32.784653 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c3f07-eca6-41af-a2a9-b48626dcf306 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.785604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c3f07-eca6-41af-a2a9-b48626dcf306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.785948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c3f07-eca6-41af-a2a9-b48626dcf306 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.786753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c3f07-eca6-41af-a2a9-b48626dcf306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:32.788814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c3f07-eca6-41af-a2a9-b48626dcf306 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:32.816881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1086/2128] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:01:32 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:32.821722 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c3f07-eca6-41af-a2a9-b48626dcf306 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:32.838222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c43de0-a339-4be2-8ded-9485e0f3e9cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:32.838840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c43de0-a339-4be2-8ded-9485e0f3e9cf service swift] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.839162 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c43de0-a339-4be2-8ded-9485e0f3e9cf service swift] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.839433 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c43de0-a339-4be2-8ded-9485e0f3e9cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.841060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c3f07-eca6-41af-a2a9-b48626dcf306 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:32.846771 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3892e-9aba-4a67-ab5c-09ab3a6a5b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:32.847469 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3892e-9aba-4a67-ab5c-09ab3a6a5b6c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.847742 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3892e-9aba-4a67-ab5c-09ab3a6a5b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.848044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3892e-9aba-4a67-ab5c-09ab3a6a5b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.848695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3892e-9aba-4a67-ab5c-09ab3a6a5b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:32.850520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3892e-9aba-4a67-ab5c-09ab3a6a5b6c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:32.863429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1087/2129] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:32.867184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3892e-9aba-4a67-ab5c-09ab3a6a5b6c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:32.881290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3892e-9aba-4a67-ab5c-09ab3a6a5b6c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:32.888366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f804459-0650-4082-961d-b81a2a40654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:32.889123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f804459-0650-4082-961d-b81a2a40654e admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.889986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f804459-0650-4082-961d-b81a2a40654e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.890363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f804459-0650-4082-961d-b81a2a40654e admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.891258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f804459-0650-4082-961d-b81a2a40654e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:32.893640 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f804459-0650-4082-961d-b81a2a40654e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:32.909932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1088/2130] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:32.913855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f804459-0650-4082-961d-b81a2a40654e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:32.929078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f804459-0650-4082-961d-b81a2a40654e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:32.934596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3c578-d42b-4dbf-b5d9-6d03546899da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:32.935320 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3c578-d42b-4dbf-b5d9-6d03546899da admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.935603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3c578-d42b-4dbf-b5d9-6d03546899da admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.935906 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3c578-d42b-4dbf-b5d9-6d03546899da admin admin] PATH_INFO: `/v3/projects/2e6cfbee34834449ba1cc2ad96f7fb4e/users/e80621a13eca4e43aaaa1079c4915acc/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:32.949211 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c43de0-a339-4be2-8ded-9485e0f3e9cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:32.952327 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c43de0-a339-4be2-8ded-9485e0f3e9cf service swift] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:32.957407 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3c578-d42b-4dbf-b5d9-6d03546899da admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6cfbee34834449ba1cc2ad96f7fb4e, user_id=e80621a13eca4e43aaaa1079c4915acc, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:32.960350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3c578-d42b-4dbf-b5d9-6d03546899da admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:32.961885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:32.977163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1089/2132] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:32 2022] PUT /identity/v3/projects/2e6cfbee34834449ba1cc2ad96f7fb4e/users/e80621a13eca4e43aaaa1079c4915acc/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:32.986823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1043/2132] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 18:01:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 18:01:32.990101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a58f65-2cc9-4405-b694-769f3dc151f4 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:32.990546 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a58f65-2cc9-4405-b694-769f3dc151f4 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:32.990868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a58f65-2cc9-4405-b694-769f3dc151f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:33.029646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c2a58f65-2cc9-4405-b694-769f3dc151f4 None None] MFA Rules not processed for user `e80621a13eca4e43aaaa1079c4915acc`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:33.031018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c2a58f65-2cc9-4405-b694-769f3dc151f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:33.157308 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1090/2133] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:33.165352 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c43de0-a339-4be2-8ded-9485e0f3e9cf service swift] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:33.167168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e2c43de0-a339-4be2-8ded-9485e0f3e9cf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:33.262500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c43de0-a339-4be2-8ded-9485e0f3e9cf service swift] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:33.346798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc0445-24e8-4a6b-a76b-d22943d0a291 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:33.347758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc0445-24e8-4a6b-a76b-d22943d0a291 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:33.348168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc0445-24e8-4a6b-a76b-d22943d0a291 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:33.348541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc0445-24e8-4a6b-a76b-d22943d0a291 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:33.359054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc0445-24e8-4a6b-a76b-d22943d0a291 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:33.360846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc0445-24e8-4a6b-a76b-d22943d0a291 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:33.373387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1044/2134] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:33.667587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a58f65-2cc9-4405-b694-769f3dc151f4 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:33.673448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:33.758244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a58f65-2cc9-4405-b694-769f3dc151f4 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:33.764236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:33.764949 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:33.765232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:33.765504 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] PATH_INFO: `/v3/users/e80621a13eca4e43aaaa1079c4915acc` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:33.766674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e80621a13eca4e43aaaa1079c4915acc)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:33.769445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:33.819143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e80621a13eca4e43aaaa1079c4915acc'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:33.819820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e80621a13eca4e43aaaa1079c4915acc'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:33.826898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e80621a13eca4e43aaaa1079c4915acc'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:33.827556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e80621a13eca4e43aaaa1079c4915acc'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:33.841541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e80621a13eca4e43aaaa1079c4915acc'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:33.855265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1091/2135] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:01:33 2022] DELETE /identity/v3/users/e80621a13eca4e43aaaa1079c4915acc => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:33.859848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2e9fe-fa6b-4bd8-a222-9362cdc3c589 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:33.860362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2e9fe-fa6b-4bd8-a222-9362cdc3c589 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:33.860479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2e9fe-fa6b-4bd8-a222-9362cdc3c589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:33.881247 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-36d2e9fe-fa6b-4bd8-a222-9362cdc3c589 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:33.882463 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-36d2e9fe-fa6b-4bd8-a222-9362cdc3c589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:33.944202 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1045/2136] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:33.954307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:33.955836 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:34.016878 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e0416-4e8b-4aed-902b-97fec14f7ab2 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:34.021241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa89c82-30d5-48ec-88b4-d7afb935d2aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:34.021874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89c82-30d5-48ec-88b4-d7afb935d2aa service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:34.022147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89c82-30d5-48ec-88b4-d7afb935d2aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:34.022414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89c82-30d5-48ec-88b4-d7afb935d2aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:34.031909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa89c82-30d5-48ec-88b4-d7afb935d2aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:34.033486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa89c82-30d5-48ec-88b4-d7afb935d2aa service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:34.043047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1092/2137] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:34.372066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2e9fe-fa6b-4bd8-a222-9362cdc3c589 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:34.373505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-36d2e9fe-fa6b-4bd8-a222-9362cdc3c589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:34.404036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2e9fe-fa6b-4bd8-a222-9362cdc3c589 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:34.415587 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:34.416303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:34.416589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:34.416849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] PATH_INFO: `/v3/projects/2e6cfbee34834449ba1cc2ad96f7fb4e` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:34.418604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6cfbee34834449ba1cc2ad96f7fb4e)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:34.420518 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:34.446936 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:34.489243 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6cfbee34834449ba1cc2ad96f7fb4e', 'request_id': 'req-7aa69494-2ed9-4845-b871-d4262af4722c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:34.497898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6cfbee34834449ba1cc2ad96f7fb4e', 'request_id': 'req-7aa69494-2ed9-4845-b871-d4262af4722c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:34.508962 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6cfbee34834449ba1cc2ad96f7fb4e', 'request_id': 'req-7aa69494-2ed9-4845-b871-d4262af4722c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:34.524108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] The token cache is being invalidate because project 2e6cfbee34834449ba1cc2ad96f7fb4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:34.524620 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:34.525697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7aa69494-2ed9-4845-b871-d4262af4722c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:34.528126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1046/2138] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:34 2022] DELETE /identity/v3/projects/2e6cfbee34834449ba1cc2ad96f7fb4e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:34.533850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb35fe5-fda0-43ae-b44b-53e6f5747617 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:34.534358 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb35fe5-fda0-43ae-b44b-53e6f5747617 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:34.534879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb35fe5-fda0-43ae-b44b-53e6f5747617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:34.566319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0eb35fe5-fda0-43ae-b44b-53e6f5747617 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:34.567988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0eb35fe5-fda0-43ae-b44b-53e6f5747617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:34.603634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:34.683577 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1093/2139] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:34.692358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63c271-b2ff-422b-a076-df41c25b512d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:34.692635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63c271-b2ff-422b-a076-df41c25b512d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:34.692913 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63c271-b2ff-422b-a076-df41c25b512d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:34.727238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-fc63c271-b2ff-422b-a076-df41c25b512d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:34.728726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-fc63c271-b2ff-422b-a076-df41c25b512d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:34.743264 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1047/2140] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:34.746715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb35fe5-fda0-43ae-b44b-53e6f5747617 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:34.757430 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb35fe5-fda0-43ae-b44b-53e6f5747617 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:34.763346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788686e-62a0-415e-9819-a110da9bc3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:34.764214 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788686e-62a0-415e-9819-a110da9bc3cc admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:34.765220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788686e-62a0-415e-9819-a110da9bc3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:34.765658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788686e-62a0-415e-9819-a110da9bc3cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:34.766479 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788686e-62a0-415e-9819-a110da9bc3cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:01:34.766777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788686e-62a0-415e-9819-a110da9bc3cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:34.769152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788686e-62a0-415e-9819-a110da9bc3cc admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:34.783558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1094/2141] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 18:01:34.788016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63c271-b2ff-422b-a076-df41c25b512d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:34.798430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63c271-b2ff-422b-a076-df41c25b512d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:34.806262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450f548-afc6-4ecb-a655-dda4acd674e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:34.807319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f548-afc6-4ecb-a655-dda4acd674e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:34.807319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f548-afc6-4ecb-a655-dda4acd674e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:34.807550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f548-afc6-4ecb-a655-dda4acd674e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:34.808226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450f548-afc6-4ecb-a655-dda4acd674e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:34.810253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450f548-afc6-4ecb-a655-dda4acd674e0 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:34.856014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1048/2142] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:34 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:34.856375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788686e-62a0-415e-9819-a110da9bc3cc admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:34.867790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788686e-62a0-415e-9819-a110da9bc3cc admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:34.930888 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5953a-007b-4897-84f8-dc462918eeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:34.931501 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5953a-007b-4897-84f8-dc462918eeab admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:34.931784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5953a-007b-4897-84f8-dc462918eeab admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:34.932207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5953a-007b-4897-84f8-dc462918eeab admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:34.932845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5953a-007b-4897-84f8-dc462918eeab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:34.935192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5953a-007b-4897-84f8-dc462918eeab admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:34.960293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1095/2143] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:01:34 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:01:34.964690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450f548-afc6-4ecb-a655-dda4acd674e0 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:34.978060 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450f548-afc6-4ecb-a655-dda4acd674e0 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:34.984812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe947bc-9583-4ce3-b01c-14e4c1be531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:34.985577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe947bc-9583-4ce3-b01c-14e4c1be531f admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:34.985691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe947bc-9583-4ce3-b01c-14e4c1be531f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:34.985942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe947bc-9583-4ce3-b01c-14e4c1be531f admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:34.987153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe947bc-9583-4ce3-b01c-14e4c1be531f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:34.988818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe947bc-9583-4ce3-b01c-14e4c1be531f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:35.005616 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1049/2144] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:35.009826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5953a-007b-4897-84f8-dc462918eeab admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:35.022199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5953a-007b-4897-84f8-dc462918eeab admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:35.028008 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bedd2-8bca-4e88-929f-76430b2adf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:35.028551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bedd2-8bca-4e88-929f-76430b2adf46 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.028831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bedd2-8bca-4e88-929f-76430b2adf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.029100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bedd2-8bca-4e88-929f-76430b2adf46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.029758 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bedd2-8bca-4e88-929f-76430b2adf46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:35.031375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bedd2-8bca-4e88-929f-76430b2adf46 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:35.048511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe947bc-9583-4ce3-b01c-14e4c1be531f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:35.049540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1096/2145] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:35.061527 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe947bc-9583-4ce3-b01c-14e4c1be531f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:35.071452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfa2dd-131d-4305-aa5e-2fb10d34d0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:35.072354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfa2dd-131d-4305-aa5e-2fb10d34d0fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.072757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfa2dd-131d-4305-aa5e-2fb10d34d0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.073152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfa2dd-131d-4305-aa5e-2fb10d34d0fb admin admin] PATH_INFO: `/v3/projects/d8069cd3a2af4ea080963d773361fea4/users/361b30fb00c4493c9e1f64d8103226c1/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.090388 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfa2dd-131d-4305-aa5e-2fb10d34d0fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8069cd3a2af4ea080963d773361fea4, user_id=361b30fb00c4493c9e1f64d8103226c1, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:35.093439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfa2dd-131d-4305-aa5e-2fb10d34d0fb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:35.111958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1050/2146] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:35 2022] PUT /identity/v3/projects/d8069cd3a2af4ea080963d773361fea4/users/361b30fb00c4493c9e1f64d8103226c1/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:35.119025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f90f5-9a10-42c9-b737-57a1559b46ee None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.119302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f90f5-9a10-42c9-b737-57a1559b46ee None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.119745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f90f5-9a10-42c9-b737-57a1559b46ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.152277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f36f90f5-9a10-42c9-b737-57a1559b46ee None None] MFA Rules not processed for user `361b30fb00c4493c9e1f64d8103226c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:35.153517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f36f90f5-9a10-42c9-b737-57a1559b46ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:35.232651 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1097/2147] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:35.237781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfa2dd-131d-4305-aa5e-2fb10d34d0fb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:35.301538 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfa2dd-131d-4305-aa5e-2fb10d34d0fb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:35.306250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d557d76-1cec-4b0a-b103-0b3d24dd8bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:35.306862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d557d76-1cec-4b0a-b103-0b3d24dd8bae admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.307137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d557d76-1cec-4b0a-b103-0b3d24dd8bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.307399 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d557d76-1cec-4b0a-b103-0b3d24dd8bae admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.308067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d557d76-1cec-4b0a-b103-0b3d24dd8bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:35.309964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d557d76-1cec-4b0a-b103-0b3d24dd8bae admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:35.342618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1051/2148] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:35 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:01:35.344050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f90f5-9a10-42c9-b737-57a1559b46ee None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:35.356325 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f90f5-9a10-42c9-b737-57a1559b46ee None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:35.375217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:35.416520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b4a3e-5568-41ea-a2dd-ca39569fea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:35.417054 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b4a3e-5568-41ea-a2dd-ca39569fea97 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.417323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b4a3e-5568-41ea-a2dd-ca39569fea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.417587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b4a3e-5568-41ea-a2dd-ca39569fea97 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.418204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b4a3e-5568-41ea-a2dd-ca39569fea97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:35.420124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b4a3e-5568-41ea-a2dd-ca39569fea97 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:35.442407 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1098/2149] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:01:35 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:35.446895 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d557d76-1cec-4b0a-b103-0b3d24dd8bae admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:35.457309 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d557d76-1cec-4b0a-b103-0b3d24dd8bae admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:35.462056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bee3b8-1437-4801-a301-41a873c5320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:35.462720 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bee3b8-1437-4801-a301-41a873c5320f admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.463087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bee3b8-1437-4801-a301-41a873c5320f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.463425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bee3b8-1437-4801-a301-41a873c5320f admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.464157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bee3b8-1437-4801-a301-41a873c5320f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:35.465824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bee3b8-1437-4801-a301-41a873c5320f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:35.477821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1052/2150] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:35.482124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b4a3e-5568-41ea-a2dd-ca39569fea97 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:35.492769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b4a3e-5568-41ea-a2dd-ca39569fea97 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:35.497592 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cc5e3-b7af-46e0-b638-98609491e6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:35.498361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cc5e3-b7af-46e0-b638-98609491e6b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.498642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cc5e3-b7af-46e0-b638-98609491e6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.498935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cc5e3-b7af-46e0-b638-98609491e6b5 admin admin] PATH_INFO: `/v3/projects/953db66a00a4408cb305b9d8a41bbfb0/users/22c7e9378c5a4b4da3955613d6797ef2/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.516508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016cc5e3-b7af-46e0-b638-98609491e6b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=953db66a00a4408cb305b9d8a41bbfb0, user_id=22c7e9378c5a4b4da3955613d6797ef2, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:35.519793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016cc5e3-b7af-46e0-b638-98609491e6b5 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:35.533755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1099/2151] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:35 2022] PUT /identity/v3/projects/953db66a00a4408cb305b9d8a41bbfb0/users/22c7e9378c5a4b4da3955613d6797ef2/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:01:35.537885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bee3b8-1437-4801-a301-41a873c5320f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:35.547979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bee3b8-1437-4801-a301-41a873c5320f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:35.560113 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:35.606844 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064fbc-2bd9-43ea-a72f-390e5c304f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:35.607439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064fbc-2bd9-43ea-a72f-390e5c304f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.607713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064fbc-2bd9-43ea-a72f-390e5c304f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.608017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064fbc-2bd9-43ea-a72f-390e5c304f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.608642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b064fbc-2bd9-43ea-a72f-390e5c304f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:35.610220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b064fbc-2bd9-43ea-a72f-390e5c304f63 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:35.620413 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1053/2152] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 18:01:35.624690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cc5e3-b7af-46e0-b638-98609491e6b5 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:35.635205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cc5e3-b7af-46e0-b638-98609491e6b5 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:35.639666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcb5cb-2807-426e-b412-4a2cfc5ac4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:35.640334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcb5cb-2807-426e-b412-4a2cfc5ac4d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.640608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcb5cb-2807-426e-b412-4a2cfc5ac4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.640874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcb5cb-2807-426e-b412-4a2cfc5ac4d2 admin admin] PATH_INFO: `/v3/projects/953db66a00a4408cb305b9d8a41bbfb0/users/22c7e9378c5a4b4da3955613d6797ef2/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.643352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcb5cb-2807-426e-b412-4a2cfc5ac4d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=953db66a00a4408cb305b9d8a41bbfb0, user_id=22c7e9378c5a4b4da3955613d6797ef2, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:35.646056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcb5cb-2807-426e-b412-4a2cfc5ac4d2 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:35.656005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1100/2153] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:35 2022] PUT /identity/v3/projects/953db66a00a4408cb305b9d8a41bbfb0/users/22c7e9378c5a4b4da3955613d6797ef2/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:35.667397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8580b-de2a-4359-abde-dc4d3c2fc307 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.667684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8580b-de2a-4359-abde-dc4d3c2fc307 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.668017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8580b-de2a-4359-abde-dc4d3c2fc307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.696499 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-bdf8580b-de2a-4359-abde-dc4d3c2fc307 None None] MFA Rules not processed for user `22c7e9378c5a4b4da3955613d6797ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:35.697701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-bdf8580b-de2a-4359-abde-dc4d3c2fc307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:35.766805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1054/2154] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:35.779545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcb5cb-2807-426e-b412-4a2cfc5ac4d2 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:35.781261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cedcb5cb-2807-426e-b412-4a2cfc5ac4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:35.838409 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcb5cb-2807-426e-b412-4a2cfc5ac4d2 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:35.842076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31669bb9-3553-4db1-8f47-3d6932cf65cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:35.842667 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31669bb9-3553-4db1-8f47-3d6932cf65cb service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.842939 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31669bb9-3553-4db1-8f47-3d6932cf65cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.843207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31669bb9-3553-4db1-8f47-3d6932cf65cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:35.852352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31669bb9-3553-4db1-8f47-3d6932cf65cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:35.853765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31669bb9-3553-4db1-8f47-3d6932cf65cb service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:35.863765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1101/2155] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:35.938844 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8580b-de2a-4359-abde-dc4d3c2fc307 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:35.949093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8580b-de2a-4359-abde-dc4d3c2fc307 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:35.952664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904dcc0-3b83-40db-a9ab-cd95fa458885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:35.953255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904dcc0-3b83-40db-a9ab-cd95fa458885 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:35.953532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904dcc0-3b83-40db-a9ab-cd95fa458885 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:35.953804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904dcc0-3b83-40db-a9ab-cd95fa458885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:36.010942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904dcc0-3b83-40db-a9ab-cd95fa458885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:36.012389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904dcc0-3b83-40db-a9ab-cd95fa458885 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:36.021481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1055/2156] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:38.757079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31669bb9-3553-4db1-8f47-3d6932cf65cb service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:38.822735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31669bb9-3553-4db1-8f47-3d6932cf65cb service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:38.827196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:38.827812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:38.828120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:38.828390 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] PATH_INFO: `/v3/users/361b30fb00c4493c9e1f64d8103226c1` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:38.829519 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=361b30fb00c4493c9e1f64d8103226c1)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:38.831639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:38.848163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:38.878508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361b30fb00c4493c9e1f64d8103226c1'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:38.879151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361b30fb00c4493c9e1f64d8103226c1'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:38.886132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361b30fb00c4493c9e1f64d8103226c1'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:38.886777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361b30fb00c4493c9e1f64d8103226c1'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:38.899549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361b30fb00c4493c9e1f64d8103226c1'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:38.913584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1102/2157] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:01:38 2022] DELETE /identity/v3/users/361b30fb00c4493c9e1f64d8103226c1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:38.917824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a4fa3-eb63-4807-8616-25c062adf868 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:38.918300 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a4fa3-eb63-4807-8616-25c062adf868 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:38.918405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a4fa3-eb63-4807-8616-25c062adf868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:38.937496 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-ad9a4fa3-eb63-4807-8616-25c062adf868 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:38.938767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ad9a4fa3-eb63-4807-8616-25c062adf868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:38.971801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:39.000824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1056/2158] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:39.011141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:39.012572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:39.069913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985011-f4d4-450f-8f47-af9821d3a2ba admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:39.073568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22b862-f7d2-4599-8cf8-56108d2d3665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:39.074161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22b862-f7d2-4599-8cf8-56108d2d3665 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:39.074433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22b862-f7d2-4599-8cf8-56108d2d3665 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:39.074698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22b862-f7d2-4599-8cf8-56108d2d3665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:39.083596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22b862-f7d2-4599-8cf8-56108d2d3665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:39.085056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22b862-f7d2-4599-8cf8-56108d2d3665 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:39.094534 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1103/2159] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:39.363865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a4fa3-eb63-4807-8616-25c062adf868 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:39.365686 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ad9a4fa3-eb63-4807-8616-25c062adf868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:39.383086 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a4fa3-eb63-4807-8616-25c062adf868 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:39.392763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:39.393407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:39.393674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:39.393937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] PATH_INFO: `/v3/projects/d8069cd3a2af4ea080963d773361fea4` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:39.395079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8069cd3a2af4ea080963d773361fea4)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:39.397039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:39.445650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8069cd3a2af4ea080963d773361fea4', 'request_id': 'req-d6eecdc1-77fa-4a2c-ba61-68378090f954'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:39.451459 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8069cd3a2af4ea080963d773361fea4', 'request_id': 'req-d6eecdc1-77fa-4a2c-ba61-68378090f954'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:39.459145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8069cd3a2af4ea080963d773361fea4', 'request_id': 'req-d6eecdc1-77fa-4a2c-ba61-68378090f954'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:39.471269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] The token cache is being invalidate because project d8069cd3a2af4ea080963d773361fea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:39.471557 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:39.472212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:39.473699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1057/2160] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:39 2022] DELETE /identity/v3/projects/d8069cd3a2af4ea080963d773361fea4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:39.478786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22b862-f7d2-4599-8cf8-56108d2d3665 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:39.489256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22b862-f7d2-4599-8cf8-56108d2d3665 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:39.554918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:39.555533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:39.555807 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:39.556103 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] PATH_INFO: `/v3/users/22c7e9378c5a4b4da3955613d6797ef2` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:39.557209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] RBAC: Authorizing `identity:delete_user(user_id=22c7e9378c5a4b4da3955613d6797ef2)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:39.559119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:39.598500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:39.605198 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22c7e9378c5a4b4da3955613d6797ef2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:39.605897 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22c7e9378c5a4b4da3955613d6797ef2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:39.613050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22c7e9378c5a4b4da3955613d6797ef2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:39.613671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22c7e9378c5a4b4da3955613d6797ef2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:39.625572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-09588323-d64e-4c6b-8c78-2e76cdcfb96e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22c7e9378c5a4b4da3955613d6797ef2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:39.637811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1104/2161] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:01:39 2022] DELETE /identity/v3/users/22c7e9378c5a4b4da3955613d6797ef2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:39.905322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:39.906839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:39.922641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eecdc1-77fa-4a2c-ba61-68378090f954 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:39.981471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:39.982098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:39.982370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:39.982637 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] PATH_INFO: `/v3/projects/953db66a00a4408cb305b9d8a41bbfb0` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:39.983800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=953db66a00a4408cb305b9d8a41bbfb0)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:39.985710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.036435 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '953db66a00a4408cb305b9d8a41bbfb0', 'request_id': 'req-347809df-1a4c-463c-b713-4ec6040e4e1d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:40.041855 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '953db66a00a4408cb305b9d8a41bbfb0', 'request_id': 'req-347809df-1a4c-463c-b713-4ec6040e4e1d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:40.044350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:40.049575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '953db66a00a4408cb305b9d8a41bbfb0', 'request_id': 'req-347809df-1a4c-463c-b713-4ec6040e4e1d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:40.062251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] The token cache is being invalidate because project 953db66a00a4408cb305b9d8a41bbfb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:40.062550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:40.063177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-347809df-1a4c-463c-b713-4ec6040e4e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:40.065427 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1058/2162] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:39 2022] DELETE /identity/v3/projects/953db66a00a4408cb305b9d8a41bbfb0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:40.070382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf672a6-bd46-4cca-a03a-d60fad2f4c6b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.070860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf672a6-bd46-4cca-a03a-d60fad2f4c6b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.070958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf672a6-bd46-4cca-a03a-d60fad2f4c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.091369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7bf672a6-bd46-4cca-a03a-d60fad2f4c6b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:40.092604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7bf672a6-bd46-4cca-a03a-d60fad2f4c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:40.151655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1105/2163] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:40.159906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74842d12-5a45-4a9e-a670-61425fb9bbf7 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.160213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74842d12-5a45-4a9e-a670-61425fb9bbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.160487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74842d12-5a45-4a9e-a670-61425fb9bbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.180131 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-74842d12-5a45-4a9e-a670-61425fb9bbf7 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:40.181328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-74842d12-5a45-4a9e-a670-61425fb9bbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:40.189208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1059/2164] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:40.193994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf672a6-bd46-4cca-a03a-d60fad2f4c6b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.203711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf672a6-bd46-4cca-a03a-d60fad2f4c6b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.208576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ef787-a960-48da-8ca4-1add36d224dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.209169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ef787-a960-48da-8ca4-1add36d224dc admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.209442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ef787-a960-48da-8ca4-1add36d224dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.209711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ef787-a960-48da-8ca4-1add36d224dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.210350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ef787-a960-48da-8ca4-1add36d224dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:01:40.210649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ef787-a960-48da-8ca4-1add36d224dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.212281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ef787-a960-48da-8ca4-1add36d224dc admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.224988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1106/2165] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:01:40.229112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74842d12-5a45-4a9e-a670-61425fb9bbf7 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.238578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74842d12-5a45-4a9e-a670-61425fb9bbf7 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.243231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0465cd7-95dd-4fa1-a9f0-be8b7ad24329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.243809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0465cd7-95dd-4fa1-a9f0-be8b7ad24329 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.244110 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0465cd7-95dd-4fa1-a9f0-be8b7ad24329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.244378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0465cd7-95dd-4fa1-a9f0-be8b7ad24329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.245001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0465cd7-95dd-4fa1-a9f0-be8b7ad24329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.246872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0465cd7-95dd-4fa1-a9f0-be8b7ad24329 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.276377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1060/2166] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:40 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:40.280405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ef787-a960-48da-8ca4-1add36d224dc admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.290403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ef787-a960-48da-8ca4-1add36d224dc admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.347309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7764e-df48-4a1d-850f-e9e7e8a03e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.347929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7764e-df48-4a1d-850f-e9e7e8a03e89 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.348205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7764e-df48-4a1d-850f-e9e7e8a03e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.348468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7764e-df48-4a1d-850f-e9e7e8a03e89 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.349101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7764e-df48-4a1d-850f-e9e7e8a03e89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.351025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7764e-df48-4a1d-850f-e9e7e8a03e89 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.373643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1107/2167] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:01:40 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:40.378383 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0465cd7-95dd-4fa1-a9f0-be8b7ad24329 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.389000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0465cd7-95dd-4fa1-a9f0-be8b7ad24329 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.393726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec5192-9fdd-4876-acce-ba5392eb2b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.394322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec5192-9fdd-4876-acce-ba5392eb2b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.394596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec5192-9fdd-4876-acce-ba5392eb2b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.394867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec5192-9fdd-4876-acce-ba5392eb2b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.395500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec5192-9fdd-4876-acce-ba5392eb2b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.397152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec5192-9fdd-4876-acce-ba5392eb2b0d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.407020 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1061/2168] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:40.411607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7764e-df48-4a1d-850f-e9e7e8a03e89 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.421831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7764e-df48-4a1d-850f-e9e7e8a03e89 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.427362 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47fa34-a164-4b92-8dc0-2bbcb3769b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.428025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47fa34-a164-4b92-8dc0-2bbcb3769b53 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.428300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47fa34-a164-4b92-8dc0-2bbcb3769b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.428563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47fa34-a164-4b92-8dc0-2bbcb3769b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.429195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47fa34-a164-4b92-8dc0-2bbcb3769b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.430785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47fa34-a164-4b92-8dc0-2bbcb3769b53 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.441474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1108/2169] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:40.445377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec5192-9fdd-4876-acce-ba5392eb2b0d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.455666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec5192-9fdd-4876-acce-ba5392eb2b0d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.460265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ec706-3a9b-4338-a901-6c28c59e205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.460898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ec706-3a9b-4338-a901-6c28c59e205f admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.461171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ec706-3a9b-4338-a901-6c28c59e205f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.461439 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ec706-3a9b-4338-a901-6c28c59e205f admin admin] PATH_INFO: `/v3/projects/ac274ea2d90a4a18965250772a6690fb/users/18e5e0c5f19347c3a230557a5423474d/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.477263 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ec706-3a9b-4338-a901-6c28c59e205f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac274ea2d90a4a18965250772a6690fb, user_id=18e5e0c5f19347c3a230557a5423474d, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.479933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ec706-3a9b-4338-a901-6c28c59e205f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.492533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1062/2170] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:40 2022] PUT /identity/v3/projects/ac274ea2d90a4a18965250772a6690fb/users/18e5e0c5f19347c3a230557a5423474d/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:01:40.502746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdff74-0625-4a11-9f5f-5dfe5b0cacda None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.503023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdff74-0625-4a11-9f5f-5dfe5b0cacda None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.503289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdff74-0625-4a11-9f5f-5dfe5b0cacda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.530763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1abdff74-0625-4a11-9f5f-5dfe5b0cacda None None] MFA Rules not processed for user `18e5e0c5f19347c3a230557a5423474d`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:40.532080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1abdff74-0625-4a11-9f5f-5dfe5b0cacda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:40.542286 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:40.600955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1109/2171] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:40.605525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ec706-3a9b-4338-a901-6c28c59e205f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.670595 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ec706-3a9b-4338-a901-6c28c59e205f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.675486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26798-df14-4568-b435-7e9d264f666f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.676116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26798-df14-4568-b435-7e9d264f666f admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.676394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26798-df14-4568-b435-7e9d264f666f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.676660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26798-df14-4568-b435-7e9d264f666f admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.677304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad26798-df14-4568-b435-7e9d264f666f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.679191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad26798-df14-4568-b435-7e9d264f666f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.705914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1063/2172] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:40 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:40.710028 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdff74-0625-4a11-9f5f-5dfe5b0cacda None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.720662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdff74-0625-4a11-9f5f-5dfe5b0cacda None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.776026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bec170-4c7f-4e47-8a8d-f78933684ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.776558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bec170-4c7f-4e47-8a8d-f78933684ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.776831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bec170-4c7f-4e47-8a8d-f78933684ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.777097 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bec170-4c7f-4e47-8a8d-f78933684ed3 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.777712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bec170-4c7f-4e47-8a8d-f78933684ed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.779618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bec170-4c7f-4e47-8a8d-f78933684ed3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.800113 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1110/2173] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:01:40 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:40.804071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26798-df14-4568-b435-7e9d264f666f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.814836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26798-df14-4568-b435-7e9d264f666f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.819582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e3957-aab5-4938-a9fc-8f2038155584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.820206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3957-aab5-4938-a9fc-8f2038155584 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.820506 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3957-aab5-4938-a9fc-8f2038155584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.820761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3957-aab5-4938-a9fc-8f2038155584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.822285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e3957-aab5-4938-a9fc-8f2038155584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.823941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e3957-aab5-4938-a9fc-8f2038155584 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.834926 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1064/2174] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:40.838808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bec170-4c7f-4e47-8a8d-f78933684ed3 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.849248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bec170-4c7f-4e47-8a8d-f78933684ed3 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.853805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23c2c-1117-4a68-924b-75cd75cc27c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.854441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23c2c-1117-4a68-924b-75cd75cc27c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.854721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23c2c-1117-4a68-924b-75cd75cc27c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.855021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23c2c-1117-4a68-924b-75cd75cc27c9 admin admin] PATH_INFO: `/v3/projects/636da381c0084623b2fea9e5c226497b/users/6fc26f138efe4ae9abb97d4451c4f895/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.870021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d23c2c-1117-4a68-924b-75cd75cc27c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636da381c0084623b2fea9e5c226497b, user_id=6fc26f138efe4ae9abb97d4451c4f895, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.872711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d23c2c-1117-4a68-924b-75cd75cc27c9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.885517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1111/2175] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:40 2022] PUT /identity/v3/projects/636da381c0084623b2fea9e5c226497b/users/6fc26f138efe4ae9abb97d4451c4f895/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:40.889438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e3957-aab5-4938-a9fc-8f2038155584 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.899291 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e3957-aab5-4938-a9fc-8f2038155584 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.959082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b57aae-ed91-416f-b336-7e38ab6899ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.959781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b57aae-ed91-416f-b336-7e38ab6899ae admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.960169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b57aae-ed91-416f-b336-7e38ab6899ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.960454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b57aae-ed91-416f-b336-7e38ab6899ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:40.961094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b57aae-ed91-416f-b336-7e38ab6899ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:40.962716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b57aae-ed91-416f-b336-7e38ab6899ae admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:40.964090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:40.977931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1065/2176] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:40.982003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23c2c-1117-4a68-924b-75cd75cc27c9 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:40.993086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23c2c-1117-4a68-924b-75cd75cc27c9 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:40.998049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1768da7-7ddb-44d5-bca5-13c538388b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:40.998702 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1768da7-7ddb-44d5-bca5-13c538388b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:40.999007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1768da7-7ddb-44d5-bca5-13c538388b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:40.999274 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1768da7-7ddb-44d5-bca5-13c538388b9e admin admin] PATH_INFO: `/v3/projects/636da381c0084623b2fea9e5c226497b/users/6fc26f138efe4ae9abb97d4451c4f895/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:41.001246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1768da7-7ddb-44d5-bca5-13c538388b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=636da381c0084623b2fea9e5c226497b, user_id=6fc26f138efe4ae9abb97d4451c4f895, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:41.004539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1768da7-7ddb-44d5-bca5-13c538388b9e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:41.015656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1112/2177] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:40 2022] PUT /identity/v3/projects/636da381c0084623b2fea9e5c226497b/users/6fc26f138efe4ae9abb97d4451c4f895/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:41.029569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46156a2-e000-498b-9d13-685d26060e2f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:41.029964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46156a2-e000-498b-9d13-685d26060e2f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:41.030242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46156a2-e000-498b-9d13-685d26060e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:41.061979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-b46156a2-e000-498b-9d13-685d26060e2f None None] MFA Rules not processed for user `6fc26f138efe4ae9abb97d4451c4f895`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:41.065919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b46156a2-e000-498b-9d13-685d26060e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:41.163715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1066/2178] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:41.171448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1768da7-7ddb-44d5-bca5-13c538388b9e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:41.173666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f1768da7-7ddb-44d5-bca5-13c538388b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:41.240834 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1768da7-7ddb-44d5-bca5-13c538388b9e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:41.244859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd10e20-4a94-4f6e-b0a5-aff3a80eb1f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:41.245468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd10e20-4a94-4f6e-b0a5-aff3a80eb1f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:41.245737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd10e20-4a94-4f6e-b0a5-aff3a80eb1f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:41.246003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd10e20-4a94-4f6e-b0a5-aff3a80eb1f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:41.255370 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd10e20-4a94-4f6e-b0a5-aff3a80eb1f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:41.256870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd10e20-4a94-4f6e-b0a5-aff3a80eb1f5 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:41.266671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1113/2179] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:41.604939 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46156a2-e000-498b-9d13-685d26060e2f None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:41.616623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46156a2-e000-498b-9d13-685d26060e2f None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:41.621945 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c8031-ff6a-45d6-8f98-d6f509fdd516 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fc26f138efe4ae9abb97d4451c4f895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '636da381c0084623b2fea9e5c226497b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:41.622556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8031-ff6a-45d6-8f98-d6f509fdd516 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:41.622896 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8031-ff6a-45d6-8f98-d6f509fdd516 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:41.623094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8031-ff6a-45d6-8f98-d6f509fdd516 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:41.623741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c8031-ff6a-45d6-8f98-d6f509fdd516 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:01:41.624072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c8031-ff6a-45d6-8f98-d6f509fdd516 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:41.625888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c8031-ff6a-45d6-8f98-d6f509fdd516 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:41.640317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1067/2180] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:01:41.642077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd10e20-4a94-4f6e-b0a5-aff3a80eb1f5 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:41.651391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd10e20-4a94-4f6e-b0a5-aff3a80eb1f5 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:41.656304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5b810-cb44-4fad-b06a-28061588d460 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fc26f138efe4ae9abb97d4451c4f895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '636da381c0084623b2fea9e5c226497b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:41.656964 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5b810-cb44-4fad-b06a-28061588d460 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:41.657324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5b810-cb44-4fad-b06a-28061588d460 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:41.657656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5b810-cb44-4fad-b06a-28061588d460 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:41.658968 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5b810-cb44-4fad-b06a-28061588d460 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:41.661018 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5b810-cb44-4fad-b06a-28061588d460 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:41.665843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:41.695638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1114/2181] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:41 2022] POST /identity/v3/projects => generated 354 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:41.696932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c8031-ff6a-45d6-8f98-d6f509fdd516 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:41.706718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c8031-ff6a-45d6-8f98-d6f509fdd516 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:41.766816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8df6a4-59ae-4c86-8af3-3aaf57d25157 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fc26f138efe4ae9abb97d4451c4f895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '636da381c0084623b2fea9e5c226497b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:41.767421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8df6a4-59ae-4c86-8af3-3aaf57d25157 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:41.767707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8df6a4-59ae-4c86-8af3-3aaf57d25157 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:41.768049 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8df6a4-59ae-4c86-8af3-3aaf57d25157 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:41.768668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8df6a4-59ae-4c86-8af3-3aaf57d25157 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:01:41.768970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8df6a4-59ae-4c86-8af3-3aaf57d25157 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:41.770585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8df6a4-59ae-4c86-8af3-3aaf57d25157 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:41.784666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1068/2182] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:01:41.823972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5b810-cb44-4fad-b06a-28061588d460 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:41.835026 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5b810-cb44-4fad-b06a-28061588d460 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:41.839709 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fc26f138efe4ae9abb97d4451c4f895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '636da381c0084623b2fea9e5c226497b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:41.840384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:41.840664 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:41.840932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] PATH_INFO: `/v3/projects/f25d9c4029d34d39b012da04c348df5e` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:41.842042 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f25d9c4029d34d39b012da04c348df5e)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:41.843967 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:41.893973 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f25d9c4029d34d39b012da04c348df5e', 'request_id': 'req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:41.900012 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f25d9c4029d34d39b012da04c348df5e', 'request_id': 'req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:41.908568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f25d9c4029d34d39b012da04c348df5e', 'request_id': 'req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:41.919759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] The token cache is being invalidate because project f25d9c4029d34d39b012da04c348df5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:41.920071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:41.920697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-7ec397c2-2cd8-4fb9-84a4-bae9eca0e4db tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:41.922172 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1115/2183] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:41 2022] DELETE /identity/v3/projects/f25d9c4029d34d39b012da04c348df5e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:42.121182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8df6a4-59ae-4c86-8af3-3aaf57d25157 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:42.123221 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7f8df6a4-59ae-4c86-8af3-3aaf57d25157 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:42.192828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8df6a4-59ae-4c86-8af3-3aaf57d25157 tempest-VolumeQuotaClassesTest-460632215 tempest-VolumeQuotaClassesTest-460632215-project-admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:42.197595 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:42.253070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:42.253711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:42.253982 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:42.254249 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] PATH_INFO: `/v3/users/18e5e0c5f19347c3a230557a5423474d` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:42.255374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e5e0c5f19347c3a230557a5423474d)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:42.257327 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:42.310847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e5e0c5f19347c3a230557a5423474d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:42.311548 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e5e0c5f19347c3a230557a5423474d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:42.319397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e5e0c5f19347c3a230557a5423474d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:42.320061 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e5e0c5f19347c3a230557a5423474d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:42.333559 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-afffdcea-5982-4f16-aa56-97a048dd9cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e5e0c5f19347c3a230557a5423474d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:42.348250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1069/2184] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:01:42 2022] DELETE /identity/v3/users/18e5e0c5f19347c3a230557a5423474d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:01:42.353316 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b940ee7-b77b-4177-b1e6-0d064fdc8568 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:42.353768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b940ee7-b77b-4177-b1e6-0d064fdc8568 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:42.353925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b940ee7-b77b-4177-b1e6-0d064fdc8568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:42.374203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2b940ee7-b77b-4177-b1e6-0d064fdc8568 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:42.375393 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2b940ee7-b77b-4177-b1e6-0d064fdc8568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:42.443237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51edd70-40cf-4ffe-a705-bb139d595fef None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:42.443590 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51edd70-40cf-4ffe-a705-bb139d595fef None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:42.443975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51edd70-40cf-4ffe-a705-bb139d595fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:42.445123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1116/2185] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:42.454570 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b940ee7-b77b-4177-b1e6-0d064fdc8568 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:42.456307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2b940ee7-b77b-4177-b1e6-0d064fdc8568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:42.464734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-e51edd70-40cf-4ffe-a705-bb139d595fef None None] MFA Rules not processed for user `3a0657648d244f12ab76c068b38e641b`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:42.465892 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e51edd70-40cf-4ffe-a705-bb139d595fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:42.513891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b940ee7-b77b-4177-b1e6-0d064fdc8568 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:42.517751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f11389-f126-43f4-bcfe-bc234609e099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:42.518368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f11389-f126-43f4-bcfe-bc234609e099 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:42.518594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f11389-f126-43f4-bcfe-bc234609e099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:42.518868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f11389-f126-43f4-bcfe-bc234609e099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:42.523208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1070/2186] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:42.528330 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f11389-f126-43f4-bcfe-bc234609e099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:42.529776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f11389-f126-43f4-bcfe-bc234609e099 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:42.540956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1117/2187] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:01:42.831249 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51edd70-40cf-4ffe-a705-bb139d595fef None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:42.832759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e51edd70-40cf-4ffe-a705-bb139d595fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:42.847081 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51edd70-40cf-4ffe-a705-bb139d595fef None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:42.851902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:42.852597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:42.852871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:42.853153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] PATH_INFO: `/v3/projects/ac274ea2d90a4a18965250772a6690fb` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:42.854293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac274ea2d90a4a18965250772a6690fb)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:42.856989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:42.891180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:42.909325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac274ea2d90a4a18965250772a6690fb', 'request_id': 'req-fb28d0da-2918-40e0-a77d-748faff69e5e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:42.916099 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac274ea2d90a4a18965250772a6690fb', 'request_id': 'req-fb28d0da-2918-40e0-a77d-748faff69e5e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:42.924393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac274ea2d90a4a18965250772a6690fb', 'request_id': 'req-fb28d0da-2918-40e0-a77d-748faff69e5e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:42.937033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] The token cache is being invalidate because project ac274ea2d90a4a18965250772a6690fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:42.937324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:42.937988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:42.940241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1071/2188] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:42 2022] DELETE /identity/v3/projects/ac274ea2d90a4a18965250772a6690fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:42.945190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f11389-f126-43f4-bcfe-bc234609e099 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:42.954763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f11389-f126-43f4-bcfe-bc234609e099 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:42.963563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:43.017155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:43.018490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:43.018755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:43.019014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] PATH_INFO: `/v3/users/6fc26f138efe4ae9abb97d4451c4f895` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:43.020178 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fc26f138efe4ae9abb97d4451c4f895)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:43.022087 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:43.071205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fc26f138efe4ae9abb97d4451c4f895'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:43.072084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fc26f138efe4ae9abb97d4451c4f895'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:43.079708 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fc26f138efe4ae9abb97d4451c4f895'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:43.080541 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fc26f138efe4ae9abb97d4451c4f895'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:43.093138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-2dbb780f-ea23-4851-a8a3-c0700b7a75bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fc26f138efe4ae9abb97d4451c4f895'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:43.106695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1118/2189] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:01:42 2022] DELETE /identity/v3/users/6fc26f138efe4ae9abb97d4451c4f895 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:01:43.418008 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:43.420113 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:43.437349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28d0da-2918-40e0-a77d-748faff69e5e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:43.505242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:43.505890 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:43.506210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:43.506422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] PATH_INFO: `/v3/projects/636da381c0084623b2fea9e5c226497b` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:43.507574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=636da381c0084623b2fea9e5c226497b)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:43.509554 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:43.559866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636da381c0084623b2fea9e5c226497b', 'request_id': 'req-6fd7781c-a19f-40b8-b6db-b1802aff26e8'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:43.566583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636da381c0084623b2fea9e5c226497b', 'request_id': 'req-6fd7781c-a19f-40b8-b6db-b1802aff26e8'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:43.575316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636da381c0084623b2fea9e5c226497b', 'request_id': 'req-6fd7781c-a19f-40b8-b6db-b1802aff26e8'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:43.589039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] The token cache is being invalidate because project 636da381c0084623b2fea9e5c226497b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:43.589403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:43.590120 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-6fd7781c-a19f-40b8-b6db-b1802aff26e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:43.591763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1072/2190] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:43 2022] DELETE /identity/v3/projects/636da381c0084623b2fea9e5c226497b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:43.596964 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec33cd-4b09-4a08-b056-114a6c139bdf None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:43.597599 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec33cd-4b09-4a08-b056-114a6c139bdf None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:43.597599 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec33cd-4b09-4a08-b056-114a6c139bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:43.620047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c4ec33cd-4b09-4a08-b056-114a6c139bdf None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:43.621520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c4ec33cd-4b09-4a08-b056-114a6c139bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:43.691946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1119/2191] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:43.699623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e78c44-c396-4549-b957-42f6da75068e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:43.699935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e78c44-c396-4549-b957-42f6da75068e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:43.700216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e78c44-c396-4549-b957-42f6da75068e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:43.721895 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-f0e78c44-c396-4549-b957-42f6da75068e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:43.723112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-f0e78c44-c396-4549-b957-42f6da75068e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:43.731449 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1073/2192] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:43.735816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec33cd-4b09-4a08-b056-114a6c139bdf None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:43.746109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec33cd-4b09-4a08-b056-114a6c139bdf None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:43.751256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280332a-f6ab-4da6-a1f9-4bf5f3d3430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:43.751850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280332a-f6ab-4da6-a1f9-4bf5f3d3430a admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:43.752676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280332a-f6ab-4da6-a1f9-4bf5f3d3430a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:43.753109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280332a-f6ab-4da6-a1f9-4bf5f3d3430a admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:43.753753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280332a-f6ab-4da6-a1f9-4bf5f3d3430a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:01:43.754049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280332a-f6ab-4da6-a1f9-4bf5f3d3430a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:43.755643 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280332a-f6ab-4da6-a1f9-4bf5f3d3430a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:43.769193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1120/2193] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:01:43.772838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e78c44-c396-4549-b957-42f6da75068e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:43.782613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e78c44-c396-4549-b957-42f6da75068e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:43.788096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6a42a-9e76-404f-9688-353a11544c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:43.788711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a42a-9e76-404f-9688-353a11544c95 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:43.788996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a42a-9e76-404f-9688-353a11544c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:43.789326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a42a-9e76-404f-9688-353a11544c95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:43.790048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6a42a-9e76-404f-9688-353a11544c95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:43.792333 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6a42a-9e76-404f-9688-353a11544c95 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:43.823417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1074/2194] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:43 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:43.827570 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280332a-f6ab-4da6-a1f9-4bf5f3d3430a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:43.838647 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280332a-f6ab-4da6-a1f9-4bf5f3d3430a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:43.852499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:43.901507 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86cecf-2018-4de3-9640-06e4e61ab532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:43.902130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86cecf-2018-4de3-9640-06e4e61ab532 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:43.902404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86cecf-2018-4de3-9640-06e4e61ab532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:43.902693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86cecf-2018-4de3-9640-06e4e61ab532 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:43.903345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86cecf-2018-4de3-9640-06e4e61ab532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:43.905297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86cecf-2018-4de3-9640-06e4e61ab532 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:43.931078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1121/2195] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:01:43 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:43.932796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6a42a-9e76-404f-9688-353a11544c95 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:43.944698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6a42a-9e76-404f-9688-353a11544c95 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:43.949641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8165917-81c3-4246-9788-f7028edce519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:43.950403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8165917-81c3-4246-9788-f7028edce519 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:43.950745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8165917-81c3-4246-9788-f7028edce519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:43.951080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8165917-81c3-4246-9788-f7028edce519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:43.951717 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8165917-81c3-4246-9788-f7028edce519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:43.953395 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8165917-81c3-4246-9788-f7028edce519 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:43.964891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1075/2196] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:43.969112 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86cecf-2018-4de3-9640-06e4e61ab532 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:43.979208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86cecf-2018-4de3-9640-06e4e61ab532 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:43.984102 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39caf43-9f7f-4179-9718-7a175050fb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:43.984741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39caf43-9f7f-4179-9718-7a175050fb20 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:43.984970 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39caf43-9f7f-4179-9718-7a175050fb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:43.985314 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39caf43-9f7f-4179-9718-7a175050fb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:43.986036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39caf43-9f7f-4179-9718-7a175050fb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:43.987952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39caf43-9f7f-4179-9718-7a175050fb20 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.004259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1122/2197] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:44.005397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8165917-81c3-4246-9788-f7028edce519 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.015815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8165917-81c3-4246-9788-f7028edce519 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.021186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef712bf-1861-49cf-bd7a-71ad898dd2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:44.021828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef712bf-1861-49cf-bd7a-71ad898dd2f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.022104 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef712bf-1861-49cf-bd7a-71ad898dd2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.022374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef712bf-1861-49cf-bd7a-71ad898dd2f5 admin admin] PATH_INFO: `/v3/projects/1fbb5c46952641cea11fbee8317f36df/users/4f533f3b6ba9408c97bf6a56bb3b0c4c/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.024550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:44.039652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef712bf-1861-49cf-bd7a-71ad898dd2f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fbb5c46952641cea11fbee8317f36df, user_id=4f533f3b6ba9408c97bf6a56bb3b0c4c, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:44.042360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef712bf-1861-49cf-bd7a-71ad898dd2f5 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.056023 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1076/2198] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:44 2022] PUT /identity/v3/projects/1fbb5c46952641cea11fbee8317f36df/users/4f533f3b6ba9408c97bf6a56bb3b0c4c/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:44.068346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca2e88-8b75-4ab6-99fe-ad5b5f2fdd2b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.068630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca2e88-8b75-4ab6-99fe-ad5b5f2fdd2b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.068981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca2e88-8b75-4ab6-99fe-ad5b5f2fdd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.096674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-91ca2e88-8b75-4ab6-99fe-ad5b5f2fdd2b None None] MFA Rules not processed for user `4f533f3b6ba9408c97bf6a56bb3b0c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:44.097873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-91ca2e88-8b75-4ab6-99fe-ad5b5f2fdd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:44.163214 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1123/2199] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:44.167947 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef712bf-1861-49cf-bd7a-71ad898dd2f5 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.229871 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef712bf-1861-49cf-bd7a-71ad898dd2f5 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.234470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c11ac-292d-4029-b724-9c1eba8dbaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:44.235065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c11ac-292d-4029-b724-9c1eba8dbaeb admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.235342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c11ac-292d-4029-b724-9c1eba8dbaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.235609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c11ac-292d-4029-b724-9c1eba8dbaeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.236311 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c11ac-292d-4029-b724-9c1eba8dbaeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:44.238543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c11ac-292d-4029-b724-9c1eba8dbaeb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.264491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1077/2200] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:44 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:44.269267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca2e88-8b75-4ab6-99fe-ad5b5f2fdd2b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.281369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca2e88-8b75-4ab6-99fe-ad5b5f2fdd2b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.347174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376026ef-3c3f-4948-9316-dd9142d0df68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:44.347820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376026ef-3c3f-4948-9316-dd9142d0df68 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.348189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376026ef-3c3f-4948-9316-dd9142d0df68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.348453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376026ef-3c3f-4948-9316-dd9142d0df68 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.349087 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376026ef-3c3f-4948-9316-dd9142d0df68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:44.351242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376026ef-3c3f-4948-9316-dd9142d0df68 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.375679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1124/2201] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:01:44 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:44.380293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c11ac-292d-4029-b724-9c1eba8dbaeb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.394330 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c11ac-292d-4029-b724-9c1eba8dbaeb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.408398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2fa3f-3d90-47e5-880a-5727bb141da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:44.414342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2fa3f-3d90-47e5-880a-5727bb141da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.414796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2fa3f-3d90-47e5-880a-5727bb141da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.415254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2fa3f-3d90-47e5-880a-5727bb141da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.416268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2fa3f-3d90-47e5-880a-5727bb141da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:44.418075 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2fa3f-3d90-47e5-880a-5727bb141da8 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.430263 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1078/2202] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:44.435011 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376026ef-3c3f-4948-9316-dd9142d0df68 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.446674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376026ef-3c3f-4948-9316-dd9142d0df68 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.451649 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a639d5-cf6b-439f-8e48-b5b39049ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:44.452366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a639d5-cf6b-439f-8e48-b5b39049ae25 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.452706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a639d5-cf6b-439f-8e48-b5b39049ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.453263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a639d5-cf6b-439f-8e48-b5b39049ae25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.453916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a639d5-cf6b-439f-8e48-b5b39049ae25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:44.456572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a639d5-cf6b-439f-8e48-b5b39049ae25 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.470233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1125/2203] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 18:01:44.472375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2fa3f-3d90-47e5-880a-5727bb141da8 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.483606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2fa3f-3d90-47e5-880a-5727bb141da8 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.489353 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0e49e-0f5d-4eca-a861-c7ca0e744d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:44.490270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e49e-0f5d-4eca-a861-c7ca0e744d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.490647 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e49e-0f5d-4eca-a861-c7ca0e744d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.490984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e49e-0f5d-4eca-a861-c7ca0e744d33 admin admin] PATH_INFO: `/v3/projects/b82ef9002b704a1890429dc04b6765aa/users/5fe78f9e17864928bca5647ca96b9b9e/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.507410 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0e49e-0f5d-4eca-a861-c7ca0e744d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b82ef9002b704a1890429dc04b6765aa, user_id=5fe78f9e17864928bca5647ca96b9b9e, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:44.510810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0e49e-0f5d-4eca-a861-c7ca0e744d33 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.527994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1079/2204] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:44 2022] PUT /identity/v3/projects/b82ef9002b704a1890429dc04b6765aa/users/5fe78f9e17864928bca5647ca96b9b9e/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:44.539430 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37206e-627d-4f99-89f9-51b8ba07854e None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.539794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37206e-627d-4f99-89f9-51b8ba07854e None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.540167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37206e-627d-4f99-89f9-51b8ba07854e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.550835 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:44.578579 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1f37206e-627d-4f99-89f9-51b8ba07854e None None] MFA Rules not processed for user `5fe78f9e17864928bca5647ca96b9b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:44.580446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1f37206e-627d-4f99-89f9-51b8ba07854e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:44.647914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1126/2205] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:44.652528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0e49e-0f5d-4eca-a861-c7ca0e744d33 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.718177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0e49e-0f5d-4eca-a861-c7ca0e744d33 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.723918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7eb02-3243-4c1b-84f5-ac510399d895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:44.724539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7eb02-3243-4c1b-84f5-ac510399d895 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.724818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7eb02-3243-4c1b-84f5-ac510399d895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.725128 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7eb02-3243-4c1b-84f5-ac510399d895 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.725778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7eb02-3243-4c1b-84f5-ac510399d895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:44.727652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7eb02-3243-4c1b-84f5-ac510399d895 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.758961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1080/2206] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:44 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:44.760103 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37206e-627d-4f99-89f9-51b8ba07854e None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.773182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37206e-627d-4f99-89f9-51b8ba07854e None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.832517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421439b-b1ee-460a-bf12-53f9cbd5320c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:44.833126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421439b-b1ee-460a-bf12-53f9cbd5320c admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.833397 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421439b-b1ee-460a-bf12-53f9cbd5320c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.833665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421439b-b1ee-460a-bf12-53f9cbd5320c admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.834298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421439b-b1ee-460a-bf12-53f9cbd5320c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:44.836252 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421439b-b1ee-460a-bf12-53f9cbd5320c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.859922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1127/2207] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:01:44 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:01:44.864270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7eb02-3243-4c1b-84f5-ac510399d895 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.874864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7eb02-3243-4c1b-84f5-ac510399d895 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.880993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c7cc1-80a7-4d72-a515-58ed010ddcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:44.881874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c7cc1-80a7-4d72-a515-58ed010ddcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.882304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c7cc1-80a7-4d72-a515-58ed010ddcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.882587 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c7cc1-80a7-4d72-a515-58ed010ddcd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.883237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c7cc1-80a7-4d72-a515-58ed010ddcd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:44.884873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c7cc1-80a7-4d72-a515-58ed010ddcd3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.896397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1081/2208] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:44.900734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421439b-b1ee-460a-bf12-53f9cbd5320c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.911052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421439b-b1ee-460a-bf12-53f9cbd5320c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.915639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba665c-08bc-473e-985a-da167e28c504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:44.916300 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba665c-08bc-473e-985a-da167e28c504 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:44.916575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba665c-08bc-473e-985a-da167e28c504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:44.916842 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba665c-08bc-473e-985a-da167e28c504 admin admin] PATH_INFO: `/v3/projects/3cf886d8afea450eb5909d9ebe866564/users/8a0d36647525497e8d9e135225e19b24/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:44.935402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba665c-08bc-473e-985a-da167e28c504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf886d8afea450eb5909d9ebe866564, user_id=8a0d36647525497e8d9e135225e19b24, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:44.938088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba665c-08bc-473e-985a-da167e28c504 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:44.952712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1128/2209] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:44 2022] PUT /identity/v3/projects/3cf886d8afea450eb5909d9ebe866564/users/8a0d36647525497e8d9e135225e19b24/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:44.957405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c7cc1-80a7-4d72-a515-58ed010ddcd3 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:44.967334 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c7cc1-80a7-4d72-a515-58ed010ddcd3 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:44.971654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:45.027629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2de28-54c2-45df-a7df-a9319a48fa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:45.028226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2de28-54c2-45df-a7df-a9319a48fa19 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:45.028500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2de28-54c2-45df-a7df-a9319a48fa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:45.028800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2de28-54c2-45df-a7df-a9319a48fa19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:45.029500 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2de28-54c2-45df-a7df-a9319a48fa19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:45.031308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2de28-54c2-45df-a7df-a9319a48fa19 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:45.043269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1082/2210] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:45.047569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba665c-08bc-473e-985a-da167e28c504 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:45.058553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba665c-08bc-473e-985a-da167e28c504 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:45.063550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36a951-cb66-4a70-8d96-ed50db6446a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:45.064246 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36a951-cb66-4a70-8d96-ed50db6446a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:45.064703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36a951-cb66-4a70-8d96-ed50db6446a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:45.065066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36a951-cb66-4a70-8d96-ed50db6446a6 admin admin] PATH_INFO: `/v3/projects/3cf886d8afea450eb5909d9ebe866564/users/8a0d36647525497e8d9e135225e19b24/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:45.067020 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36a951-cb66-4a70-8d96-ed50db6446a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf886d8afea450eb5909d9ebe866564, user_id=8a0d36647525497e8d9e135225e19b24, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:45.069828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36a951-cb66-4a70-8d96-ed50db6446a6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:45.081149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1129/2211] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:45 2022] PUT /identity/v3/projects/3cf886d8afea450eb5909d9ebe866564/users/8a0d36647525497e8d9e135225e19b24/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:45.092088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863ebba-eb88-420b-9970-c597b8ab0b92 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:45.092444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863ebba-eb88-420b-9970-c597b8ab0b92 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:45.092846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863ebba-eb88-420b-9970-c597b8ab0b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:45.126007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-9863ebba-eb88-420b-9970-c597b8ab0b92 None None] MFA Rules not processed for user `8a0d36647525497e8d9e135225e19b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:45.127814 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9863ebba-eb88-420b-9970-c597b8ab0b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:45.214139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1083/2212] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:01:45.225852 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36a951-cb66-4a70-8d96-ed50db6446a6 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:45.227716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2a36a951-cb66-4a70-8d96-ed50db6446a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:45.286387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:45.292362 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36a951-cb66-4a70-8d96-ed50db6446a6 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:45.297877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef3253-effc-474c-9a75-012785b625fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:45.298565 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef3253-effc-474c-9a75-012785b625fd service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:45.299743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef3253-effc-474c-9a75-012785b625fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:45.299743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef3253-effc-474c-9a75-012785b625fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:45.309861 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef3253-effc-474c-9a75-012785b625fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:45.311014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef3253-effc-474c-9a75-012785b625fd service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:45.321305 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1130/2213] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:45.368732 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863ebba-eb88-420b-9970-c597b8ab0b92 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:45.370307 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9863ebba-eb88-420b-9970-c597b8ab0b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:45.385091 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863ebba-eb88-420b-9970-c597b8ab0b92 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:45.397037 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00584b72-263c-4ace-8a74-f4a176e2285b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:45.397667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00584b72-263c-4ace-8a74-f4a176e2285b service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:45.397952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00584b72-263c-4ace-8a74-f4a176e2285b service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:45.398226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00584b72-263c-4ace-8a74-f4a176e2285b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:45.470054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00584b72-263c-4ace-8a74-f4a176e2285b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:45.471776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00584b72-263c-4ace-8a74-f4a176e2285b service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:45.483049 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1084/2214] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:48.405279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef3253-effc-474c-9a75-012785b625fd service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:48.466675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef3253-effc-474c-9a75-012785b625fd service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:48.471174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:48.471782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:48.472082 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:48.472352 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] PATH_INFO: `/v3/users/4f533f3b6ba9408c97bf6a56bb3b0c4c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:48.473457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f533f3b6ba9408c97bf6a56bb3b0c4c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:48.475372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:48.520775 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f533f3b6ba9408c97bf6a56bb3b0c4c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:48.521347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f533f3b6ba9408c97bf6a56bb3b0c4c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:48.528120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f533f3b6ba9408c97bf6a56bb3b0c4c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:48.528737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f533f3b6ba9408c97bf6a56bb3b0c4c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:48.540461 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f533f3b6ba9408c97bf6a56bb3b0c4c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:48.553652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1131/2215] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:01:48 2022] DELETE /identity/v3/users/4f533f3b6ba9408c97bf6a56bb3b0c4c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:48.558098 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b9bca-8d0e-463f-a0a6-e1494b6c422d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:48.558561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b9bca-8d0e-463f-a0a6-e1494b6c422d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:48.558683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b9bca-8d0e-463f-a0a6-e1494b6c422d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:48.577989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-054b9bca-8d0e-463f-a0a6-e1494b6c422d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:48.579193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-054b9bca-8d0e-463f-a0a6-e1494b6c422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:48.599945 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:48.639093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1085/2216] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:48.649369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:48.650752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:48.709638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acbf5c-aae0-4fd6-9aca-00aefa4810fb admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:48.713294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6b88f-3c33-4afe-8062-d4d85bd08a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:48.713890 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6b88f-3c33-4afe-8062-d4d85bd08a06 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:48.714166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6b88f-3c33-4afe-8062-d4d85bd08a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:48.714434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6b88f-3c33-4afe-8062-d4d85bd08a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:48.723492 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6b88f-3c33-4afe-8062-d4d85bd08a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:48.724942 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6b88f-3c33-4afe-8062-d4d85bd08a06 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:48.734237 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1132/2217] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:01:49.010109 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b9bca-8d0e-463f-a0a6-e1494b6c422d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:49.012051 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-054b9bca-8d0e-463f-a0a6-e1494b6c422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:49.026895 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b9bca-8d0e-463f-a0a6-e1494b6c422d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:49.031904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:49.032552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:49.032854 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:49.033129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] PATH_INFO: `/v3/projects/1fbb5c46952641cea11fbee8317f36df` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:49.034945 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fbb5c46952641cea11fbee8317f36df)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:49.036930 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:49.094690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fbb5c46952641cea11fbee8317f36df', 'request_id': 'req-de8ad3de-371a-4af7-b52d-817b78935806'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.102016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fbb5c46952641cea11fbee8317f36df', 'request_id': 'req-de8ad3de-371a-4af7-b52d-817b78935806'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.113949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fbb5c46952641cea11fbee8317f36df', 'request_id': 'req-de8ad3de-371a-4af7-b52d-817b78935806'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.126693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] The token cache is being invalidate because project 1fbb5c46952641cea11fbee8317f36df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:49.126975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.127659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.129678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1086/2218] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:49 2022] DELETE /identity/v3/projects/1fbb5c46952641cea11fbee8317f36df => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:49.133449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6b88f-3c33-4afe-8062-d4d85bd08a06 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:49.146299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6b88f-3c33-4afe-8062-d4d85bd08a06 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:49.155675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:49.214134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:49.214847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:49.215200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:49.215567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] PATH_INFO: `/v3/users/5fe78f9e17864928bca5647ca96b9b9e` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:49.216795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe78f9e17864928bca5647ca96b9b9e)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:49.218825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:49.274819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe78f9e17864928bca5647ca96b9b9e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.275539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe78f9e17864928bca5647ca96b9b9e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.282802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe78f9e17864928bca5647ca96b9b9e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.284077 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe78f9e17864928bca5647ca96b9b9e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.296340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe78f9e17864928bca5647ca96b9b9e'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.310744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1133/2219] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:01:49 2022] DELETE /identity/v3/users/5fe78f9e17864928bca5647ca96b9b9e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:49.590103 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:49.592022 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:49.606552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ad3de-371a-4af7-b52d-817b78935806 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:49.667096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:49.667729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:49.668045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:49.668317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] PATH_INFO: `/v3/projects/b82ef9002b704a1890429dc04b6765aa` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:49.669417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b82ef9002b704a1890429dc04b6765aa)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:49.671322 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:49.717823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b82ef9002b704a1890429dc04b6765aa', 'request_id': 'req-21e1858a-e70d-49f1-8c59-de81bfedc45a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.719787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:49.724840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b82ef9002b704a1890429dc04b6765aa', 'request_id': 'req-21e1858a-e70d-49f1-8c59-de81bfedc45a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.732229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b82ef9002b704a1890429dc04b6765aa', 'request_id': 'req-21e1858a-e70d-49f1-8c59-de81bfedc45a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.743593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] The token cache is being invalidate because project b82ef9002b704a1890429dc04b6765aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:49.743899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.744525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.746088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1087/2220] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:49 2022] DELETE /identity/v3/projects/b82ef9002b704a1890429dc04b6765aa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:01:49.750164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:49.751587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:49.764598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bed53d-80ca-495d-a898-a09ca4f5fd83 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:49.821130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:49.821752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:49.822027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:49.822296 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] PATH_INFO: `/v3/users/8a0d36647525497e8d9e135225e19b24` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:49.823401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a0d36647525497e8d9e135225e19b24)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:49.825361 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:49.859201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:49.874445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a0d36647525497e8d9e135225e19b24'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.875081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a0d36647525497e8d9e135225e19b24'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.882750 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a0d36647525497e8d9e135225e19b24'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.883375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a0d36647525497e8d9e135225e19b24'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.896379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5cecb67a-a281-4387-913a-8266db892870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a0d36647525497e8d9e135225e19b24'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:49.908929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1134/2221] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:01:49 2022] DELETE /identity/v3/users/8a0d36647525497e8d9e135225e19b24 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:50.180864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:50.182366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:50.195622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1858a-e70d-49f1-8c59-de81bfedc45a admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:50.253185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:50.253808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.254084 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.254350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] PATH_INFO: `/v3/projects/3cf886d8afea450eb5909d9ebe866564` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.255445 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf886d8afea450eb5909d9ebe866564)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:50.257367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:50.306024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf886d8afea450eb5909d9ebe866564', 'request_id': 'req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:50.311617 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf886d8afea450eb5909d9ebe866564', 'request_id': 'req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:50.320162 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf886d8afea450eb5909d9ebe866564', 'request_id': 'req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:50.332150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] The token cache is being invalidate because project 3cf886d8afea450eb5909d9ebe866564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:01:50.332452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:50.333100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a81cce2-fe9b-4bdb-a55b-2d5b2c8572ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:01:50.334605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1088/2222] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:01:50 2022] DELETE /identity/v3/projects/3cf886d8afea450eb5909d9ebe866564 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:50.340502 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10e36e-7b2b-43cb-bdf5-3b1a2c6afdb5 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.340780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10e36e-7b2b-43cb-bdf5-3b1a2c6afdb5 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.341046 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10e36e-7b2b-43cb-bdf5-3b1a2c6afdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.361934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7d10e36e-7b2b-43cb-bdf5-3b1a2c6afdb5 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:50.363231 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7d10e36e-7b2b-43cb-bdf5-3b1a2c6afdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:50.426113 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1135/2223] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:50.435600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae03cb-79e8-429a-b6e2-035ef3adbf4d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.435904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae03cb-79e8-429a-b6e2-035ef3adbf4d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.436216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae03cb-79e8-429a-b6e2-035ef3adbf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.457254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-4bae03cb-79e8-429a-b6e2-035ef3adbf4d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:50.458460 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-4bae03cb-79e8-429a-b6e2-035ef3adbf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:50.466616 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1089/2224] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:01:50.471379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10e36e-7b2b-43cb-bdf5-3b1a2c6afdb5 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:50.481093 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10e36e-7b2b-43cb-bdf5-3b1a2c6afdb5 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:50.485590 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d12e18-70d0-4a45-aee2-c3f2b4cdd9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:50.486180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d12e18-70d0-4a45-aee2-c3f2b4cdd9bc admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.486480 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d12e18-70d0-4a45-aee2-c3f2b4cdd9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.487365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d12e18-70d0-4a45-aee2-c3f2b4cdd9bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.488044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d12e18-70d0-4a45-aee2-c3f2b4cdd9bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:01:50.488345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d12e18-70d0-4a45-aee2-c3f2b4cdd9bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:50.489980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d12e18-70d0-4a45-aee2-c3f2b4cdd9bc admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:50.501388 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1136/2225] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:01:50.505752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae03cb-79e8-429a-b6e2-035ef3adbf4d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:50.514819 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae03cb-79e8-429a-b6e2-035ef3adbf4d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:50.519832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959ceda-86a5-4ceb-9b89-c172218f522e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:50.520441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ceda-86a5-4ceb-9b89-c172218f522e admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.520712 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ceda-86a5-4ceb-9b89-c172218f522e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.520977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ceda-86a5-4ceb-9b89-c172218f522e admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.521600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959ceda-86a5-4ceb-9b89-c172218f522e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:50.523459 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959ceda-86a5-4ceb-9b89-c172218f522e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:50.551835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1090/2226] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:01:50 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:50.556115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d12e18-70d0-4a45-aee2-c3f2b4cdd9bc admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:50.566514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d12e18-70d0-4a45-aee2-c3f2b4cdd9bc admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:50.629293 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b1c61-7db4-4889-ac5b-3e43e8853d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:50.629914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b1c61-7db4-4889-ac5b-3e43e8853d22 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.630223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b1c61-7db4-4889-ac5b-3e43e8853d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.630451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b1c61-7db4-4889-ac5b-3e43e8853d22 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.631061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b1c61-7db4-4889-ac5b-3e43e8853d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:50.632986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b1c61-7db4-4889-ac5b-3e43e8853d22 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:50.654563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1137/2227] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:01:50 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:01:50.659438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959ceda-86a5-4ceb-9b89-c172218f522e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:50.670025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959ceda-86a5-4ceb-9b89-c172218f522e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:50.675321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa17da-c713-415d-a0c1-8a8d67bca3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:50.675939 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa17da-c713-415d-a0c1-8a8d67bca3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.676213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa17da-c713-415d-a0c1-8a8d67bca3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.676482 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa17da-c713-415d-a0c1-8a8d67bca3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.677112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa17da-c713-415d-a0c1-8a8d67bca3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:50.678809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa17da-c713-415d-a0c1-8a8d67bca3e0 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:50.688666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1091/2228] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:50.693128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b1c61-7db4-4889-ac5b-3e43e8853d22 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:50.703282 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b1c61-7db4-4889-ac5b-3e43e8853d22 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:50.708025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e1b2a-67ca-42bc-86bc-4bcbb04c562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:50.708617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e1b2a-67ca-42bc-86bc-4bcbb04c562d admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.708889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e1b2a-67ca-42bc-86bc-4bcbb04c562d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.709157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e1b2a-67ca-42bc-86bc-4bcbb04c562d admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.709793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e1b2a-67ca-42bc-86bc-4bcbb04c562d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:50.711382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e1b2a-67ca-42bc-86bc-4bcbb04c562d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:50.722526 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1138/2229] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:01:50.726666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa17da-c713-415d-a0c1-8a8d67bca3e0 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:50.736367 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa17da-c713-415d-a0c1-8a8d67bca3e0 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:50.741496 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b767831-eedc-43f0-94b4-ae0df007ad4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:50.742127 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b767831-eedc-43f0-94b4-ae0df007ad4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.742400 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b767831-eedc-43f0-94b4-ae0df007ad4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.742666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b767831-eedc-43f0-94b4-ae0df007ad4e admin admin] PATH_INFO: `/v3/projects/7e6ce392c40f4e60970ca429ffc1c24e/users/8ad35d90a75442f0b01fbbf028435c12/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.757206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b767831-eedc-43f0-94b4-ae0df007ad4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6ce392c40f4e60970ca429ffc1c24e, user_id=8ad35d90a75442f0b01fbbf028435c12, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:50.759813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b767831-eedc-43f0-94b4-ae0df007ad4e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:50.773044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1092/2230] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:01:50 2022] PUT /identity/v3/projects/7e6ce392c40f4e60970ca429ffc1c24e/users/8ad35d90a75442f0b01fbbf028435c12/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:01:50.785049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be67bc3-8a4c-4386-81a8-24f50d132e46 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.785577 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be67bc3-8a4c-4386-81a8-24f50d132e46 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.785678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be67bc3-8a4c-4386-81a8-24f50d132e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.814410 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-3be67bc3-8a4c-4386-81a8-24f50d132e46 None None] MFA Rules not processed for user `8ad35d90a75442f0b01fbbf028435c12`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:01:50.815720 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3be67bc3-8a4c-4386-81a8-24f50d132e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:50.819585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:50.881068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1139/2231] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:01:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:01:50.894021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b767831-eedc-43f0-94b4-ae0df007ad4e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:01:50.895689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5b767831-eedc-43f0-94b4-ae0df007ad4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:01:50.905673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:01:50.954627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b767831-eedc-43f0-94b4-ae0df007ad4e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:01:50.959047 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4968eea-a5db-42f0-977b-ad522332d1df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae3566d85c9c4c559ba672269ab38f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:01:50.959675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4968eea-a5db-42f0-977b-ad522332d1df service glance] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:01:50.959980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4968eea-a5db-42f0-977b-ad522332d1df service glance] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:01:50.960252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4968eea-a5db-42f0-977b-ad522332d1df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:01:50.968959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4968eea-a5db-42f0-977b-ad522332d1df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:01:50.970373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4968eea-a5db-42f0-977b-ad522332d1df service glance] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:01:50.980381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1093/2232] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:01:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:02:00.878743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=25 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:00.976271 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=23 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:18.870587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be67bc3-8a4c-4386-81a8-24f50d132e46 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:18.932440 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be67bc3-8a4c-4386-81a8-24f50d132e46 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:18.936887 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:18.937507 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:18.937783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:18.938054 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] PATH_INFO: `/v3/users/8ad35d90a75442f0b01fbbf028435c12` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:18.939161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad35d90a75442f0b01fbbf028435c12)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:18.941110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:18.988263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad35d90a75442f0b01fbbf028435c12'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:18.988934 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad35d90a75442f0b01fbbf028435c12'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:18.996151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad35d90a75442f0b01fbbf028435c12'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:18.996774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad35d90a75442f0b01fbbf028435c12'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:19.008735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad35d90a75442f0b01fbbf028435c12'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:19.021739 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1140/2233] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:02:18 2022] DELETE /identity/v3/users/8ad35d90a75442f0b01fbbf028435c12 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:19.026609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1dd96-fa9b-4c00-8c54-c27a5f23053f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:19.027108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1dd96-fa9b-4c00-8c54-c27a5f23053f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:19.027240 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1dd96-fa9b-4c00-8c54-c27a5f23053f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:19.045759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1ae1dd96-fa9b-4c00-8c54-c27a5f23053f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:19.046949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1ae1dd96-fa9b-4c00-8c54-c27a5f23053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:19.108499 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1094/2234] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:19.119289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:19.120853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:19.180804 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da89ea-1d0d-4914-93bc-adc52862e8ff admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:19.184432 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52a42b-ab48-4f97-b9b9-17d0817cf14c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:19.184959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52a42b-ab48-4f97-b9b9-17d0817cf14c service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:19.185229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52a42b-ab48-4f97-b9b9-17d0817cf14c service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:19.185492 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52a42b-ab48-4f97-b9b9-17d0817cf14c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:19.195180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52a42b-ab48-4f97-b9b9-17d0817cf14c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:19.196634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52a42b-ab48-4f97-b9b9-17d0817cf14c service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:19.205816 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1141/2235] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:02:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:02:19.473752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1dd96-fa9b-4c00-8c54-c27a5f23053f None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:19.475737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1ae1dd96-fa9b-4c00-8c54-c27a5f23053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:19.489288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1dd96-fa9b-4c00-8c54-c27a5f23053f None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:19.495145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:19.496393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:19.496667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:19.496943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] PATH_INFO: `/v3/projects/7e6ce392c40f4e60970ca429ffc1c24e` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:19.498258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6ce392c40f4e60970ca429ffc1c24e)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:19.500187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:19.552999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6ce392c40f4e60970ca429ffc1c24e', 'request_id': 'req-af9dda07-ff32-4b57-89b7-950ea44336c6'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:19.559509 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6ce392c40f4e60970ca429ffc1c24e', 'request_id': 'req-af9dda07-ff32-4b57-89b7-950ea44336c6'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:19.575370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6ce392c40f4e60970ca429ffc1c24e', 'request_id': 'req-af9dda07-ff32-4b57-89b7-950ea44336c6'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:19.588108 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] The token cache is being invalidate because project 7e6ce392c40f4e60970ca429ffc1c24e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:02:19.588469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:19.589168 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af9dda07-ff32-4b57-89b7-950ea44336c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:19.590729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1095/2236] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:02:19 2022] DELETE /identity/v3/projects/7e6ce392c40f4e60970ca429ffc1c24e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:19.595595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5fe9b-c75c-4147-bc72-d2f1e8247881 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:19.595896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5fe9b-c75c-4147-bc72-d2f1e8247881 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:19.596163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5fe9b-c75c-4147-bc72-d2f1e8247881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:19.619473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-1dc5fe9b-c75c-4147-bc72-d2f1e8247881 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:19.620685 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1dc5fe9b-c75c-4147-bc72-d2f1e8247881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:19.638337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:19.693824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1142/2237] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:19.757534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f67d9-9054-4b94-8cd5-9438148f372d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:19.757768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f67d9-9054-4b94-8cd5-9438148f372d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:19.758048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f67d9-9054-4b94-8cd5-9438148f372d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:19.778993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-e28f67d9-9054-4b94-8cd5-9438148f372d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:19.780218 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e28f67d9-9054-4b94-8cd5-9438148f372d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:19.788734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1096/2238] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:19.792916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5fe9b-c75c-4147-bc72-d2f1e8247881 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:19.803230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5fe9b-c75c-4147-bc72-d2f1e8247881 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:19.807823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a74796-e7ed-4c47-89c1-e4c197271a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:19.808455 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a74796-e7ed-4c47-89c1-e4c197271a51 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:19.808725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a74796-e7ed-4c47-89c1-e4c197271a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:19.808986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a74796-e7ed-4c47-89c1-e4c197271a51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:19.809613 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a74796-e7ed-4c47-89c1-e4c197271a51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:02:19.809905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a74796-e7ed-4c47-89c1-e4c197271a51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:19.811479 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a74796-e7ed-4c47-89c1-e4c197271a51 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:19.822990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1143/2239] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:02:19.827644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f67d9-9054-4b94-8cd5-9438148f372d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:19.837494 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f67d9-9054-4b94-8cd5-9438148f372d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:19.842224 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff07d6-58ce-475a-832f-07ee04d2bd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:19.842822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff07d6-58ce-475a-832f-07ee04d2bd48 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:19.843096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff07d6-58ce-475a-832f-07ee04d2bd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:19.843366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff07d6-58ce-475a-832f-07ee04d2bd48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:19.844034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff07d6-58ce-475a-832f-07ee04d2bd48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:19.845933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff07d6-58ce-475a-832f-07ee04d2bd48 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:19.878419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1097/2240] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:02:19 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:02:19.879475 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a74796-e7ed-4c47-89c1-e4c197271a51 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:19.889407 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a74796-e7ed-4c47-89c1-e4c197271a51 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:19.945629 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a2c2f-3a31-488b-9799-0d3a5571a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:19.946253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a2c2f-3a31-488b-9799-0d3a5571a038 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:19.946531 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a2c2f-3a31-488b-9799-0d3a5571a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:19.946801 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a2c2f-3a31-488b-9799-0d3a5571a038 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:19.947420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a2c2f-3a31-488b-9799-0d3a5571a038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:19.949390 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a2c2f-3a31-488b-9799-0d3a5571a038 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:19.970114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1144/2241] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:02:19 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:02:19.974753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff07d6-58ce-475a-832f-07ee04d2bd48 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:19.984884 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff07d6-58ce-475a-832f-07ee04d2bd48 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:19.989377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6db73-343c-4297-9d44-eb0714a7ce6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:19.990527 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6db73-343c-4297-9d44-eb0714a7ce6c admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:19.990800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6db73-343c-4297-9d44-eb0714a7ce6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:19.991069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6db73-343c-4297-9d44-eb0714a7ce6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:19.991736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6db73-343c-4297-9d44-eb0714a7ce6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:19.993340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6db73-343c-4297-9d44-eb0714a7ce6c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:20.003519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1098/2242] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:02:20.007340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a2c2f-3a31-488b-9799-0d3a5571a038 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:20.017280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a2c2f-3a31-488b-9799-0d3a5571a038 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:20.022375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2440ec3-3a05-431c-b9f6-25727bca23c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:20.022962 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2440ec3-3a05-431c-b9f6-25727bca23c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:20.023239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2440ec3-3a05-431c-b9f6-25727bca23c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:20.023508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2440ec3-3a05-431c-b9f6-25727bca23c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:20.024191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2440ec3-3a05-431c-b9f6-25727bca23c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:20.025912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2440ec3-3a05-431c-b9f6-25727bca23c6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:20.036464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1145/2243] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:02:20.041424 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6db73-343c-4297-9d44-eb0714a7ce6c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:20.051555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6db73-343c-4297-9d44-eb0714a7ce6c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:20.056300 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60170459-ef19-4ce4-b869-49f7f16c62e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:20.056941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170459-ef19-4ce4-b869-49f7f16c62e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:20.057513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170459-ef19-4ce4-b869-49f7f16c62e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:20.058041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170459-ef19-4ce4-b869-49f7f16c62e7 admin admin] PATH_INFO: `/v3/projects/60b43e37ed324531a0aaa707bb7e5246/users/0c3a3e79e7cb466bbfba014759e6fa0b/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:20.072313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60170459-ef19-4ce4-b869-49f7f16c62e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b43e37ed324531a0aaa707bb7e5246, user_id=0c3a3e79e7cb466bbfba014759e6fa0b, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:20.074919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60170459-ef19-4ce4-b869-49f7f16c62e7 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:20.087904 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1099/2244] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:02:20 2022] PUT /identity/v3/projects/60b43e37ed324531a0aaa707bb7e5246/users/0c3a3e79e7cb466bbfba014759e6fa0b/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:02:20.097866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8f376-13c3-4ab4-9e85-5973a44468f0 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:20.098153 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8f376-13c3-4ab4-9e85-5973a44468f0 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:20.098425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8f376-13c3-4ab4-9e85-5973a44468f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:20.126075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-94c8f376-13c3-4ab4-9e85-5973a44468f0 None None] MFA Rules not processed for user `0c3a3e79e7cb466bbfba014759e6fa0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:20.127375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-94c8f376-13c3-4ab4-9e85-5973a44468f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:20.191946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1146/2245] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:20.203481 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60170459-ef19-4ce4-b869-49f7f16c62e7 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:20.205180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-60170459-ef19-4ce4-b869-49f7f16c62e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:20.228096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:20.263973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60170459-ef19-4ce4-b869-49f7f16c62e7 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:20.267631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ef016-44aa-4c37-884b-cc960ed6f536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:20.268179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ef016-44aa-4c37-884b-cc960ed6f536 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:20.268450 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ef016-44aa-4c37-884b-cc960ed6f536 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:20.268713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ef016-44aa-4c37-884b-cc960ed6f536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:20.277685 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ef016-44aa-4c37-884b-cc960ed6f536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:20.279101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ef016-44aa-4c37-884b-cc960ed6f536 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:20.288245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1100/2246] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:02:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:02:20.817087 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8f376-13c3-4ab4-9e85-5973a44468f0 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:20.819710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-94c8f376-13c3-4ab4-9e85-5973a44468f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:20.840024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:20.879669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8f376-13c3-4ab4-9e85-5973a44468f0 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:20.884064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aad2d-c9e1-4aa2-b4dd-4423737c7fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:20.884680 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aad2d-c9e1-4aa2-b4dd-4423737c7fce service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:20.884955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aad2d-c9e1-4aa2-b4dd-4423737c7fce service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:20.885222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aad2d-c9e1-4aa2-b4dd-4423737c7fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:20.886932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-136aad2d-c9e1-4aa2-b4dd-4423737c7fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:20.948950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136aad2d-c9e1-4aa2-b4dd-4423737c7fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:20.950417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136aad2d-c9e1-4aa2-b4dd-4423737c7fce service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:20.960163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1147/2247] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:02:24.240277 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ef016-44aa-4c37-884b-cc960ed6f536 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:24.241833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5d3ef016-44aa-4c37-884b-cc960ed6f536 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:24.256579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ef016-44aa-4c37-884b-cc960ed6f536 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:24.261064 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:24.261676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:24.261949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:24.262242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] PATH_INFO: `/v3/users/b9187955c3cf4b77ba86e67a6f378e6a` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:24.263358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9187955c3cf4b77ba86e67a6f378e6a)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:24.265308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:24.315691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9187955c3cf4b77ba86e67a6f378e6a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:24.316384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9187955c3cf4b77ba86e67a6f378e6a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:24.324312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9187955c3cf4b77ba86e67a6f378e6a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:24.324964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9187955c3cf4b77ba86e67a6f378e6a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:24.338320 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b9b5be32-09b2-4b64-9812-49515cf0657a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9187955c3cf4b77ba86e67a6f378e6a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:24.353440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1101/2248] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:02:24 2022] DELETE /identity/v3/users/b9187955c3cf4b77ba86e67a6f378e6a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:24.533021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aad2d-c9e1-4aa2-b4dd-4423737c7fce service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:24.546747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aad2d-c9e1-4aa2-b4dd-4423737c7fce service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:24.616126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:24.616754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:24.617029 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:24.617340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] PATH_INFO: `/v3/projects/261d62defa42436cbcc5e712c62b76f4` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:24.618420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=261d62defa42436cbcc5e712c62b76f4)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:24.620413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:24.673251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '261d62defa42436cbcc5e712c62b76f4', 'request_id': 'req-32080c7b-9d46-46e6-8b12-46b9a4398e55'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:24.680063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '261d62defa42436cbcc5e712c62b76f4', 'request_id': 'req-32080c7b-9d46-46e6-8b12-46b9a4398e55'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:24.686843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '261d62defa42436cbcc5e712c62b76f4', 'request_id': 'req-32080c7b-9d46-46e6-8b12-46b9a4398e55'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:24.695083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:24.700184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] The token cache is being invalidate because project 261d62defa42436cbcc5e712c62b76f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:02:24.700465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:24.701164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32080c7b-9d46-46e6-8b12-46b9a4398e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:24.703159 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1148/2249] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:02:24 2022] DELETE /identity/v3/projects/261d62defa42436cbcc5e712c62b76f4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:24.708079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc233e1-2593-4b2c-bbbc-e0e3fbea0920 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:24.708361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc233e1-2593-4b2c-bbbc-e0e3fbea0920 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:24.708631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc233e1-2593-4b2c-bbbc-e0e3fbea0920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:24.728187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-5dc233e1-2593-4b2c-bbbc-e0e3fbea0920 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:24.729333 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5dc233e1-2593-4b2c-bbbc-e0e3fbea0920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:24.791870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1102/2250] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:24.799700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed3d00-859c-4848-9972-5521569ff272 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:24.800030 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed3d00-859c-4848-9972-5521569ff272 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:24.800302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed3d00-859c-4848-9972-5521569ff272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:24.823449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-36ed3d00-859c-4848-9972-5521569ff272 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:24.824903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-36ed3d00-859c-4848-9972-5521569ff272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:24.833780 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1149/2251] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:24.838089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc233e1-2593-4b2c-bbbc-e0e3fbea0920 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:24.848710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc233e1-2593-4b2c-bbbc-e0e3fbea0920 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:24.854014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930ce67-8c7f-4a56-a9df-da7c593bc5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:24.854644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930ce67-8c7f-4a56-a9df-da7c593bc5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:24.854921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930ce67-8c7f-4a56-a9df-da7c593bc5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:24.855195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930ce67-8c7f-4a56-a9df-da7c593bc5a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:24.855866 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930ce67-8c7f-4a56-a9df-da7c593bc5a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:02:24.856193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930ce67-8c7f-4a56-a9df-da7c593bc5a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:24.857815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930ce67-8c7f-4a56-a9df-da7c593bc5a8 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:24.870054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1103/2252] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:02:24.874578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed3d00-859c-4848-9972-5521569ff272 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:24.883939 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed3d00-859c-4848-9972-5521569ff272 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:24.888506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04539a45-9f79-4eda-a864-76e634293982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:24.889091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04539a45-9f79-4eda-a864-76e634293982 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:24.889366 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04539a45-9f79-4eda-a864-76e634293982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:24.889635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04539a45-9f79-4eda-a864-76e634293982 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:24.890248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04539a45-9f79-4eda-a864-76e634293982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:24.892171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04539a45-9f79-4eda-a864-76e634293982 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:24.918356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1150/2253] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:02:24 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:02:24.922375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930ce67-8c7f-4a56-a9df-da7c593bc5a8 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:24.933669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930ce67-8c7f-4a56-a9df-da7c593bc5a8 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:24.952394 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:24.991846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5e01f-1539-44f6-b821-18bb512605ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:24.992393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5e01f-1539-44f6-b821-18bb512605ae admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:24.992671 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5e01f-1539-44f6-b821-18bb512605ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:24.992966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5e01f-1539-44f6-b821-18bb512605ae admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:24.993586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5e01f-1539-44f6-b821-18bb512605ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:24.995486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5e01f-1539-44f6-b821-18bb512605ae admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:25.015463 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1104/2254] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:02:24 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:02:25.019636 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04539a45-9f79-4eda-a864-76e634293982 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:25.030314 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04539a45-9f79-4eda-a864-76e634293982 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:25.034967 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09172-97ef-4532-8b02-a784d83e85bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:25.035564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09172-97ef-4532-8b02-a784d83e85bf admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:25.035861 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09172-97ef-4532-8b02-a784d83e85bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:25.036156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09172-97ef-4532-8b02-a784d83e85bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:25.037462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09172-97ef-4532-8b02-a784d83e85bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:25.038925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09172-97ef-4532-8b02-a784d83e85bf admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:25.049273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1151/2255] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:02:25.054104 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5e01f-1539-44f6-b821-18bb512605ae admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:25.063847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5e01f-1539-44f6-b821-18bb512605ae admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:25.068694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80244c-5687-41bb-a8ef-6a1dabc1915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:25.069291 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80244c-5687-41bb-a8ef-6a1dabc1915a admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:25.069564 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80244c-5687-41bb-a8ef-6a1dabc1915a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:25.069828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80244c-5687-41bb-a8ef-6a1dabc1915a admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:25.070457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80244c-5687-41bb-a8ef-6a1dabc1915a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:25.072056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80244c-5687-41bb-a8ef-6a1dabc1915a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:25.082422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1105/2256] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:02:25.086575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09172-97ef-4532-8b02-a784d83e85bf admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:25.096289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09172-97ef-4532-8b02-a784d83e85bf admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:25.101299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d072d-9c0e-4e88-9655-18fe4d4591b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:25.102397 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d072d-9c0e-4e88-9655-18fe4d4591b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:25.102680 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d072d-9c0e-4e88-9655-18fe4d4591b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:25.102956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d072d-9c0e-4e88-9655-18fe4d4591b3 admin admin] PATH_INFO: `/v3/projects/73482f491b764ee6806628485c909f60/users/709cff6c77f24c8d95c42121ee776876/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:25.118173 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d072d-9c0e-4e88-9655-18fe4d4591b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73482f491b764ee6806628485c909f60, user_id=709cff6c77f24c8d95c42121ee776876, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:25.120954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d072d-9c0e-4e88-9655-18fe4d4591b3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:25.133804 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1152/2257] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:02:25 2022] PUT /identity/v3/projects/73482f491b764ee6806628485c909f60/users/709cff6c77f24c8d95c42121ee776876/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:25.144664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d2b03-1608-45c3-ae9a-8dd0b0d1ff53 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:25.145132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d2b03-1608-45c3-ae9a-8dd0b0d1ff53 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:25.145366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d2b03-1608-45c3-ae9a-8dd0b0d1ff53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:25.173269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-231d2b03-1608-45c3-ae9a-8dd0b0d1ff53 None None] MFA Rules not processed for user `709cff6c77f24c8d95c42121ee776876`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:25.174456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-231d2b03-1608-45c3-ae9a-8dd0b0d1ff53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:25.238873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1106/2258] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:25.249443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d072d-9c0e-4e88-9655-18fe4d4591b3 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:25.251160 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ba4d072d-9c0e-4e88-9655-18fe4d4591b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:25.308972 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d072d-9c0e-4e88-9655-18fe4d4591b3 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:25.312554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6015c-35af-4de9-95e5-003ed0372fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:25.313137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6015c-35af-4de9-95e5-003ed0372fec service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:25.313405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6015c-35af-4de9-95e5-003ed0372fec service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:25.313665 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6015c-35af-4de9-95e5-003ed0372fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:25.322450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6015c-35af-4de9-95e5-003ed0372fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:25.323905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6015c-35af-4de9-95e5-003ed0372fec service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:25.332719 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1153/2259] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:02:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:02:35.236033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:35.328706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:35.845224 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d2b03-1608-45c3-ae9a-8dd0b0d1ff53 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:35.908848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d2b03-1608-45c3-ae9a-8dd0b0d1ff53 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:35.914425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:35.915039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:35.915314 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:35.915579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] PATH_INFO: `/v3/users/709cff6c77f24c8d95c42121ee776876` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:35.916721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] RBAC: Authorizing `identity:delete_user(user_id=709cff6c77f24c8d95c42121ee776876)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:35.918616 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:35.968516 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '709cff6c77f24c8d95c42121ee776876'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:35.969164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '709cff6c77f24c8d95c42121ee776876'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:35.977279 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '709cff6c77f24c8d95c42121ee776876'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:35.977968 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '709cff6c77f24c8d95c42121ee776876'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:35.989964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '709cff6c77f24c8d95c42121ee776876'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:36.004255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1107/2260] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:02:35 2022] DELETE /identity/v3/users/709cff6c77f24c8d95c42121ee776876 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:36.009191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c2f82-717d-4342-892e-3b4b04ecd8a5 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:36.009473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c2f82-717d-4342-892e-3b4b04ecd8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:36.009744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c2f82-717d-4342-892e-3b4b04ecd8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:36.029845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7f5c2f82-717d-4342-892e-3b4b04ecd8a5 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:36.031029 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7f5c2f82-717d-4342-892e-3b4b04ecd8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:36.093937 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1154/2261] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:36.104841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:36.106351 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:36.168402 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdc210-72f0-49c9-b285-c89414efa00c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:36.172068 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4bac3-d140-419e-af70-e58c1f8db6b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:36.172674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4bac3-d140-419e-af70-e58c1f8db6b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:36.172948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4bac3-d140-419e-af70-e58c1f8db6b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:36.173215 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4bac3-d140-419e-af70-e58c1f8db6b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:36.182745 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4bac3-d140-419e-af70-e58c1f8db6b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:36.184210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4bac3-d140-419e-af70-e58c1f8db6b1 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:36.194691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1108/2262] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:02:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:02:36.485824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c2f82-717d-4342-892e-3b4b04ecd8a5 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:36.487313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7f5c2f82-717d-4342-892e-3b4b04ecd8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:36.503838 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c2f82-717d-4342-892e-3b4b04ecd8a5 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:36.509978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:36.510633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:36.510909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:36.511259 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] PATH_INFO: `/v3/projects/73482f491b764ee6806628485c909f60` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:36.512382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73482f491b764ee6806628485c909f60)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:36.514350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:36.561850 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73482f491b764ee6806628485c909f60', 'request_id': 'req-a3a480a8-c6d4-48bf-bbab-796590d76008'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:36.567151 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73482f491b764ee6806628485c909f60', 'request_id': 'req-a3a480a8-c6d4-48bf-bbab-796590d76008'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:36.575017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73482f491b764ee6806628485c909f60', 'request_id': 'req-a3a480a8-c6d4-48bf-bbab-796590d76008'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:36.586960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] The token cache is being invalidate because project 73482f491b764ee6806628485c909f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:02:36.587251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:36.587943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a3a480a8-c6d4-48bf-bbab-796590d76008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:36.589459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1155/2263] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:02:36 2022] DELETE /identity/v3/projects/73482f491b764ee6806628485c909f60 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:36.594669 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd7edc-1948-4db6-9e6a-3558e791fe0f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:36.595126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd7edc-1948-4db6-9e6a-3558e791fe0f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:36.595259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd7edc-1948-4db6-9e6a-3558e791fe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:36.614851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-e5dd7edc-1948-4db6-9e6a-3558e791fe0f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:36.616640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e5dd7edc-1948-4db6-9e6a-3558e791fe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:36.634065 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:36.676878 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1109/2264] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:36.685570 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e03f2a-f56c-4449-a325-49440d3598c1 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:36.685854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e03f2a-f56c-4449-a325-49440d3598c1 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:36.686117 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e03f2a-f56c-4449-a325-49440d3598c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:36.706956 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-d1e03f2a-f56c-4449-a325-49440d3598c1 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:36.708170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d1e03f2a-f56c-4449-a325-49440d3598c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:36.716524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1156/2265] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:36.720781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd7edc-1948-4db6-9e6a-3558e791fe0f None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:36.730604 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd7edc-1948-4db6-9e6a-3558e791fe0f None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:36.735201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d7da0-2945-48aa-8074-b0acf32963a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:36.735792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d7da0-2945-48aa-8074-b0acf32963a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:36.736096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d7da0-2945-48aa-8074-b0acf32963a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:36.736364 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d7da0-2945-48aa-8074-b0acf32963a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:36.736993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d7da0-2945-48aa-8074-b0acf32963a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:02:36.737289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d7da0-2945-48aa-8074-b0acf32963a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:36.738864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d7da0-2945-48aa-8074-b0acf32963a8 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:36.750783 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1110/2266] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:02:36.754899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e03f2a-f56c-4449-a325-49440d3598c1 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:36.764687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e03f2a-f56c-4449-a325-49440d3598c1 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:36.769499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b1af2-10cf-45bd-ab1e-15fca344891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:36.770119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b1af2-10cf-45bd-ab1e-15fca344891d admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:36.770391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b1af2-10cf-45bd-ab1e-15fca344891d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:36.770652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b1af2-10cf-45bd-ab1e-15fca344891d admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:36.771292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b1af2-10cf-45bd-ab1e-15fca344891d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:36.774456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b1af2-10cf-45bd-ab1e-15fca344891d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:36.807297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1157/2267] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:02:36 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:02:36.811723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d7da0-2945-48aa-8074-b0acf32963a8 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:36.822105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d7da0-2945-48aa-8074-b0acf32963a8 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:36.884033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e432a3e-e60a-411c-a064-20a9fbba5489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:36.884633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e432a3e-e60a-411c-a064-20a9fbba5489 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:36.884910 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e432a3e-e60a-411c-a064-20a9fbba5489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:36.885182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e432a3e-e60a-411c-a064-20a9fbba5489 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:36.885804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e432a3e-e60a-411c-a064-20a9fbba5489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:36.887817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e432a3e-e60a-411c-a064-20a9fbba5489 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:36.910938 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1111/2268] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:02:36 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:02:36.915150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b1af2-10cf-45bd-ab1e-15fca344891d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:36.925165 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b1af2-10cf-45bd-ab1e-15fca344891d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:36.930025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040d8f-b6ba-4548-9fc2-9a59287aa1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:36.930608 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040d8f-b6ba-4548-9fc2-9a59287aa1f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:36.930879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040d8f-b6ba-4548-9fc2-9a59287aa1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:36.931140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040d8f-b6ba-4548-9fc2-9a59287aa1f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:36.931767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb040d8f-b6ba-4548-9fc2-9a59287aa1f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:36.933386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb040d8f-b6ba-4548-9fc2-9a59287aa1f2 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:36.943086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1158/2269] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:02:36.947955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e432a3e-e60a-411c-a064-20a9fbba5489 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:36.958381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e432a3e-e60a-411c-a064-20a9fbba5489 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:36.963958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5d63d-aed1-47c8-a432-91e6635a817c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:36.964566 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5d63d-aed1-47c8-a432-91e6635a817c admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:36.964823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5d63d-aed1-47c8-a432-91e6635a817c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:36.965089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5d63d-aed1-47c8-a432-91e6635a817c admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:36.965774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5d63d-aed1-47c8-a432-91e6635a817c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:36.967382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5d63d-aed1-47c8-a432-91e6635a817c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:36.981269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1112/2270] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:02:36.985502 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040d8f-b6ba-4548-9fc2-9a59287aa1f2 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:36.995711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040d8f-b6ba-4548-9fc2-9a59287aa1f2 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:37.000219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e02d7c-9026-4a94-8c43-b5999e9d4362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:37.000845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02d7c-9026-4a94-8c43-b5999e9d4362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.001116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02d7c-9026-4a94-8c43-b5999e9d4362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.001382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02d7c-9026-4a94-8c43-b5999e9d4362 admin admin] PATH_INFO: `/v3/projects/70f7134104124f489332a919cb2a5a3f/users/50cb67eec9d9445e87fb62bf0fadb907/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.016457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e02d7c-9026-4a94-8c43-b5999e9d4362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f7134104124f489332a919cb2a5a3f, user_id=50cb67eec9d9445e87fb62bf0fadb907, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:37.019099 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e02d7c-9026-4a94-8c43-b5999e9d4362 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:37.032191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1159/2271] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:02:36 2022] PUT /identity/v3/projects/70f7134104124f489332a919cb2a5a3f/users/50cb67eec9d9445e87fb62bf0fadb907/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:37.042555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a89d9-ac07-4c2e-bfdc-4836a77ca271 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.042835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a89d9-ac07-4c2e-bfdc-4836a77ca271 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.043102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a89d9-ac07-4c2e-bfdc-4836a77ca271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.074493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-458a89d9-ac07-4c2e-bfdc-4836a77ca271 None None] MFA Rules not processed for user `50cb67eec9d9445e87fb62bf0fadb907`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:37.075625 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-458a89d9-ac07-4c2e-bfdc-4836a77ca271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:37.138338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1113/2272] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:37.143113 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e02d7c-9026-4a94-8c43-b5999e9d4362 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:37.167327 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:37.210563 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e02d7c-9026-4a94-8c43-b5999e9d4362 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:37.220616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66dde-d563-4864-8635-8c552f2bc1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:37.221225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66dde-d563-4864-8635-8c552f2bc1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.221500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66dde-d563-4864-8635-8c552f2bc1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.221768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66dde-d563-4864-8635-8c552f2bc1e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.222438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66dde-d563-4864-8635-8c552f2bc1e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:37.224401 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66dde-d563-4864-8635-8c552f2bc1e0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:37.254473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1160/2273] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:02:37 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:02:37.259541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a89d9-ac07-4c2e-bfdc-4836a77ca271 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:37.270294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a89d9-ac07-4c2e-bfdc-4836a77ca271 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:37.282306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:37.329275 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba477011-56f8-4ea6-abc4-ec7e7b26d724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:37.329882 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba477011-56f8-4ea6-abc4-ec7e7b26d724 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.330250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba477011-56f8-4ea6-abc4-ec7e7b26d724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.330726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba477011-56f8-4ea6-abc4-ec7e7b26d724 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.331462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba477011-56f8-4ea6-abc4-ec7e7b26d724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:37.333464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba477011-56f8-4ea6-abc4-ec7e7b26d724 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:37.355759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1114/2274] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:02:37 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:02:37.359842 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66dde-d563-4864-8635-8c552f2bc1e0 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:37.371147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66dde-d563-4864-8635-8c552f2bc1e0 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:37.375737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c581f-4067-443a-9c4b-65316b3901d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:37.376364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c581f-4067-443a-9c4b-65316b3901d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.376638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c581f-4067-443a-9c4b-65316b3901d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.376902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c581f-4067-443a-9c4b-65316b3901d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.377531 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c581f-4067-443a-9c4b-65316b3901d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:37.379128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c581f-4067-443a-9c4b-65316b3901d9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:37.390108 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1161/2275] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:02:37.394354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba477011-56f8-4ea6-abc4-ec7e7b26d724 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:37.404524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba477011-56f8-4ea6-abc4-ec7e7b26d724 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:37.409109 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d890990-8220-4e0c-94aa-3f251cf6ced2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:37.409739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d890990-8220-4e0c-94aa-3f251cf6ced2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.410019 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d890990-8220-4e0c-94aa-3f251cf6ced2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.410292 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d890990-8220-4e0c-94aa-3f251cf6ced2 admin admin] PATH_INFO: `/v3/projects/885fa5cd06a44c07b234833bfa548e4c/users/836b50398d364da3b1cc1cd82bcd3fa2/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.427981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d890990-8220-4e0c-94aa-3f251cf6ced2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=885fa5cd06a44c07b234833bfa548e4c, user_id=836b50398d364da3b1cc1cd82bcd3fa2, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:37.431230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d890990-8220-4e0c-94aa-3f251cf6ced2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:37.443789 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1115/2276] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:02:37 2022] PUT /identity/v3/projects/885fa5cd06a44c07b234833bfa548e4c/users/836b50398d364da3b1cc1cd82bcd3fa2/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:37.448753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c581f-4067-443a-9c4b-65316b3901d9 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:37.458811 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c581f-4067-443a-9c4b-65316b3901d9 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:37.515957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d072d12-8dfc-416f-8942-71a725e593c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:37.516634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d072d12-8dfc-416f-8942-71a725e593c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.516826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d072d12-8dfc-416f-8942-71a725e593c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.517100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d072d12-8dfc-416f-8942-71a725e593c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.517745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d072d12-8dfc-416f-8942-71a725e593c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:37.519371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d072d12-8dfc-416f-8942-71a725e593c7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:37.530713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1162/2277] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:02:37.534984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d890990-8220-4e0c-94aa-3f251cf6ced2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:37.545145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d890990-8220-4e0c-94aa-3f251cf6ced2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:37.549633 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb288c-2eda-465a-86e4-47c6cfbd5110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:37.550265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb288c-2eda-465a-86e4-47c6cfbd5110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.550541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb288c-2eda-465a-86e4-47c6cfbd5110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.550810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb288c-2eda-465a-86e4-47c6cfbd5110 admin admin] PATH_INFO: `/v3/projects/885fa5cd06a44c07b234833bfa548e4c/users/836b50398d364da3b1cc1cd82bcd3fa2/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.552961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb288c-2eda-465a-86e4-47c6cfbd5110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=885fa5cd06a44c07b234833bfa548e4c, user_id=836b50398d364da3b1cc1cd82bcd3fa2, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:37.556125 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb288c-2eda-465a-86e4-47c6cfbd5110 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:37.566124 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1116/2278] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:02:37 2022] PUT /identity/v3/projects/885fa5cd06a44c07b234833bfa548e4c/users/836b50398d364da3b1cc1cd82bcd3fa2/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:37.578160 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8041b-dbbf-48ca-9f0a-8a71ddb42c31 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.578517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8041b-dbbf-48ca-9f0a-8a71ddb42c31 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.578874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8041b-dbbf-48ca-9f0a-8a71ddb42c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.608783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-d6d8041b-dbbf-48ca-9f0a-8a71ddb42c31 None None] MFA Rules not processed for user `836b50398d364da3b1cc1cd82bcd3fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:37.609914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d6d8041b-dbbf-48ca-9f0a-8a71ddb42c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:37.677474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1163/2279] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:37.690974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb288c-2eda-465a-86e4-47c6cfbd5110 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:37.692814 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-65bb288c-2eda-465a-86e4-47c6cfbd5110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:37.750988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb288c-2eda-465a-86e4-47c6cfbd5110 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:37.754692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d6e13-2a36-4155-bc57-37ce233e1e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:37.755293 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6e13-2a36-4155-bc57-37ce233e1e26 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.755594 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6e13-2a36-4155-bc57-37ce233e1e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.755859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6e13-2a36-4155-bc57-37ce233e1e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.764847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d6e13-2a36-4155-bc57-37ce233e1e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:37.766272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d6e13-2a36-4155-bc57-37ce233e1e26 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:37.777566 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1117/2280] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:02:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:02:37.856155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8041b-dbbf-48ca-9f0a-8a71ddb42c31 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:37.865751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8041b-dbbf-48ca-9f0a-8a71ddb42c31 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:37.870009 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812ea1f-b5ac-48a1-a2be-bae7466ce2f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:37.870595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812ea1f-b5ac-48a1-a2be-bae7466ce2f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:37.870864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812ea1f-b5ac-48a1-a2be-bae7466ce2f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:37.871129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812ea1f-b5ac-48a1-a2be-bae7466ce2f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:37.883522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:37.938299 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812ea1f-b5ac-48a1-a2be-bae7466ce2f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:37.939752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812ea1f-b5ac-48a1-a2be-bae7466ce2f4 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:37.948695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1164/2281] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:02:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:02:47.772750 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:47.944617 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=21 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:57.816337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d6e13-2a36-4155-bc57-37ce233e1e26 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:57.817838 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-2a9d6e13-2a36-4155-bc57-37ce233e1e26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:57.884242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d6e13-2a36-4155-bc57-37ce233e1e26 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:57.889506 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:57.890142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:57.890408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:57.890676 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] PATH_INFO: `/v3/users/0c3a3e79e7cb466bbfba014759e6fa0b` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:57.891803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c3a3e79e7cb466bbfba014759e6fa0b)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:57.893763 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:57.943389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c3a3e79e7cb466bbfba014759e6fa0b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:57.945349 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c3a3e79e7cb466bbfba014759e6fa0b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:57.951777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c3a3e79e7cb466bbfba014759e6fa0b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:57.952516 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c3a3e79e7cb466bbfba014759e6fa0b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:57.965038 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-51d45a85-d169-42cf-82eb-17c1f906f5e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c3a3e79e7cb466bbfba014759e6fa0b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:57.977863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1118/2282] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:02:57 2022] DELETE /identity/v3/users/0c3a3e79e7cb466bbfba014759e6fa0b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:57.983191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22c3dd-fe03-4127-bfa4-349f4adb6f86 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:57.983703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22c3dd-fe03-4127-bfa4-349f4adb6f86 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:57.983859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22c3dd-fe03-4127-bfa4-349f4adb6f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.003856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7b22c3dd-fe03-4127-bfa4-349f4adb6f86 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:58.005072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7b22c3dd-fe03-4127-bfa4-349f4adb6f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:58.041271 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1602c9-04cf-44c4-9075-e34545aed98f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:58.041603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1602c9-04cf-44c4-9075-e34545aed98f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:58.041928 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1602c9-04cf-44c4-9075-e34545aed98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.062532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-fb1602c9-04cf-44c4-9075-e34545aed98f None None] MFA Rules not processed for user `3a0657648d244f12ab76c068b38e641b`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:58.063761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-fb1602c9-04cf-44c4-9075-e34545aed98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:58.088427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1165/2283] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:58.098941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22c3dd-fe03-4127-bfa4-349f4adb6f86 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:58.100368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7b22c3dd-fe03-4127-bfa4-349f4adb6f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:58.124955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1119/2284] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:58.162084 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22c3dd-fe03-4127-bfa4-349f4adb6f86 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:58.166131 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02685ce-5f89-4528-8bcb-21f6755abdd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:58.167144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02685ce-5f89-4528-8bcb-21f6755abdd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:58.167428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02685ce-5f89-4528-8bcb-21f6755abdd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:58.167724 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02685ce-5f89-4528-8bcb-21f6755abdd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.178350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02685ce-5f89-4528-8bcb-21f6755abdd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:58.179787 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02685ce-5f89-4528-8bcb-21f6755abdd3 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:58.197193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1166/2285] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:02:58.492950 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1602c9-04cf-44c4-9075-e34545aed98f None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:58.494422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-fb1602c9-04cf-44c4-9075-e34545aed98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:58.509272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1602c9-04cf-44c4-9075-e34545aed98f None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:58.513822 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:58.514469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:58.514752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:58.515006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] PATH_INFO: `/v3/projects/60b43e37ed324531a0aaa707bb7e5246` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.516169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b43e37ed324531a0aaa707bb7e5246)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:58.518137 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:58.540438 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:58.570039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b43e37ed324531a0aaa707bb7e5246', 'request_id': 'req-c2767959-a3d2-466b-b280-5eb664f8046a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:58.576891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b43e37ed324531a0aaa707bb7e5246', 'request_id': 'req-c2767959-a3d2-466b-b280-5eb664f8046a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:58.586372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b43e37ed324531a0aaa707bb7e5246', 'request_id': 'req-c2767959-a3d2-466b-b280-5eb664f8046a'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:58.599257 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] The token cache is being invalidate because project 60b43e37ed324531a0aaa707bb7e5246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:02:58.599474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:58.600162 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c2767959-a3d2-466b-b280-5eb664f8046a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:02:58.601652 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1120/2286] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:02:58 2022] DELETE /identity/v3/projects/60b43e37ed324531a0aaa707bb7e5246 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:02:58.606926 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024fe57-e372-4330-b4e3-ca1174b9a619 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:58.607212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024fe57-e372-4330-b4e3-ca1174b9a619 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:58.607481 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024fe57-e372-4330-b4e3-ca1174b9a619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.627095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2024fe57-e372-4330-b4e3-ca1174b9a619 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:58.628325 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2024fe57-e372-4330-b4e3-ca1174b9a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:58.689909 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1167/2287] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:58.699470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80666782-6ab8-4895-8aa2-a45ad52d03d9 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:58.699748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80666782-6ab8-4895-8aa2-a45ad52d03d9 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:58.700646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80666782-6ab8-4895-8aa2-a45ad52d03d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.721068 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-80666782-6ab8-4895-8aa2-a45ad52d03d9 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:58.722350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-80666782-6ab8-4895-8aa2-a45ad52d03d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:58.731282 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1121/2288] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:58.735842 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024fe57-e372-4330-b4e3-ca1174b9a619 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:58.745944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024fe57-e372-4330-b4e3-ca1174b9a619 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:58.750829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7de569-d478-4642-95d9-22225f8ba35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:58.751425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7de569-d478-4642-95d9-22225f8ba35a admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:58.751722 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7de569-d478-4642-95d9-22225f8ba35a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:58.752031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7de569-d478-4642-95d9-22225f8ba35a admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.752671 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7de569-d478-4642-95d9-22225f8ba35a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:02:58.752965 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7de569-d478-4642-95d9-22225f8ba35a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:58.754548 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7de569-d478-4642-95d9-22225f8ba35a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:58.771920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1168/2289] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:02:58.773784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80666782-6ab8-4895-8aa2-a45ad52d03d9 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:58.783524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80666782-6ab8-4895-8aa2-a45ad52d03d9 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:58.789031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac1067-27b1-4b27-9ce0-4feffec49a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:58.789715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac1067-27b1-4b27-9ce0-4feffec49a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:58.790057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac1067-27b1-4b27-9ce0-4feffec49a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:58.790398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac1067-27b1-4b27-9ce0-4feffec49a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.791103 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac1067-27b1-4b27-9ce0-4feffec49a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:58.793601 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac1067-27b1-4b27-9ce0-4feffec49a9c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:58.832060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7de569-d478-4642-95d9-22225f8ba35a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:58.832219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1122/2290] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:02:58 2022] POST /identity/v3/projects => generated 399 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:02:58.846375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7de569-d478-4642-95d9-22225f8ba35a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:58.881782 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:02:58.910588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c8ef7-8c6a-4002-a7d6-9ddf5345f77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:58.911193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c8ef7-8c6a-4002-a7d6-9ddf5345f77f admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:58.911748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c8ef7-8c6a-4002-a7d6-9ddf5345f77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:58.911748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c8ef7-8c6a-4002-a7d6-9ddf5345f77f admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.912357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c8ef7-8c6a-4002-a7d6-9ddf5345f77f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:58.914322 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c8ef7-8c6a-4002-a7d6-9ddf5345f77f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:58.936769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1169/2291] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:02:58 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:02:58.940643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac1067-27b1-4b27-9ce0-4feffec49a9c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:58.950847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac1067-27b1-4b27-9ce0-4feffec49a9c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:58.955435 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b185699-269e-4f91-8108-72189230aacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:58.956055 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b185699-269e-4f91-8108-72189230aacb admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:58.956331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b185699-269e-4f91-8108-72189230aacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:58.956598 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b185699-269e-4f91-8108-72189230aacb admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.957236 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b185699-269e-4f91-8108-72189230aacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:58.958821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b185699-269e-4f91-8108-72189230aacb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:58.969517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1123/2292] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:02:58.973381 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c8ef7-8c6a-4002-a7d6-9ddf5345f77f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:58.984166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c8ef7-8c6a-4002-a7d6-9ddf5345f77f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:58.988950 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed345cbe-7b29-4708-8d1b-ae8f4aed535e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:58.989583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed345cbe-7b29-4708-8d1b-ae8f4aed535e admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:58.989860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed345cbe-7b29-4708-8d1b-ae8f4aed535e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:58.990170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed345cbe-7b29-4708-8d1b-ae8f4aed535e admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:58.990829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed345cbe-7b29-4708-8d1b-ae8f4aed535e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:58.992519 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed345cbe-7b29-4708-8d1b-ae8f4aed535e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:59.006399 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1170/2293] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 18:02:59.011290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b185699-269e-4f91-8108-72189230aacb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:59.022092 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b185699-269e-4f91-8108-72189230aacb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:59.027557 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5516d9-712d-4913-be33-b67312bb7d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:59.028298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5516d9-712d-4913-be33-b67312bb7d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:59.028643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5516d9-712d-4913-be33-b67312bb7d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:59.028983 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5516d9-712d-4913-be33-b67312bb7d4f admin admin] PATH_INFO: `/v3/projects/09ce7d67e5fd43f2a59e314104937d28/users/1d85679510ff4784ae6b9d0177761c9b/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:59.044833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5516d9-712d-4913-be33-b67312bb7d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ce7d67e5fd43f2a59e314104937d28, user_id=1d85679510ff4784ae6b9d0177761c9b, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:59.047632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5516d9-712d-4913-be33-b67312bb7d4f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:59.061262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1124/2294] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:02:59 2022] PUT /identity/v3/projects/09ce7d67e5fd43f2a59e314104937d28/users/1d85679510ff4784ae6b9d0177761c9b/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:02:59.071935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6e798-e7af-4a04-93a3-838f829e9bef None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:59.072224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6e798-e7af-4a04-93a3-838f829e9bef None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:59.072494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6e798-e7af-4a04-93a3-838f829e9bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:59.102074 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-6ec6e798-e7af-4a04-93a3-838f829e9bef None None] MFA Rules not processed for user `1d85679510ff4784ae6b9d0177761c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:02:59.103245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6ec6e798-e7af-4a04-93a3-838f829e9bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:59.176706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1171/2295] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:02:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:02:59.188996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5516d9-712d-4913-be33-b67312bb7d4f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:02:59.190786 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8e5516d9-712d-4913-be33-b67312bb7d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:02:59.251094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5516d9-712d-4913-be33-b67312bb7d4f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:02:59.254742 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd0215-e375-499e-9325-f7bdcfefcef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:02:59.255377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd0215-e375-499e-9325-f7bdcfefcef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:02:59.255658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd0215-e375-499e-9325-f7bdcfefcef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:02:59.255955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd0215-e375-499e-9325-f7bdcfefcef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:02:59.265657 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd0215-e375-499e-9325-f7bdcfefcef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:02:59.267153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd0215-e375-499e-9325-f7bdcfefcef5 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:02:59.276555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1125/2296] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:02:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:09.173447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:09.272161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:12.799783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6e798-e7af-4a04-93a3-838f829e9bef None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:12.902415 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6e798-e7af-4a04-93a3-838f829e9bef None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:12.907209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:12.907823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:12.908267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:12.908532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] PATH_INFO: `/v3/users/1d85679510ff4784ae6b9d0177761c9b` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:12.909704 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d85679510ff4784ae6b9d0177761c9b)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:12.911676 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:12.971748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d85679510ff4784ae6b9d0177761c9b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:12.972433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d85679510ff4784ae6b9d0177761c9b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:12.979255 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d85679510ff4784ae6b9d0177761c9b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:12.979902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d85679510ff4784ae6b9d0177761c9b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:12.992598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d85679510ff4784ae6b9d0177761c9b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:13.006797 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1172/2297] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:03:12 2022] DELETE /identity/v3/users/1d85679510ff4784ae6b9d0177761c9b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:13.013166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d38af-acb9-4c4e-b464-a59670e7e2ee None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:13.013166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d38af-acb9-4c4e-b464-a59670e7e2ee None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:13.013166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d38af-acb9-4c4e-b464-a59670e7e2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:13.069875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-298d38af-acb9-4c4e-b464-a59670e7e2ee None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:13.071127 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-298d38af-acb9-4c4e-b464-a59670e7e2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:13.135987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1126/2298] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:03:13.147050 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:13.148703 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:13.192488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:13.234161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffddb70-0c30-4fe4-9399-81127d7db622 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:13.237794 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fac649-c463-4c32-8731-42f747ccdb9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:13.238391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fac649-c463-4c32-8731-42f747ccdb9a service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:13.238670 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fac649-c463-4c32-8731-42f747ccdb9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:13.238943 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fac649-c463-4c32-8731-42f747ccdb9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:13.248553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fac649-c463-4c32-8731-42f747ccdb9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:13.249984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fac649-c463-4c32-8731-42f747ccdb9a service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:13.259463 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1173/2299] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:03:13.572900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d38af-acb9-4c4e-b464-a59670e7e2ee None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:13.575301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-298d38af-acb9-4c4e-b464-a59670e7e2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:13.587907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d38af-acb9-4c4e-b464-a59670e7e2ee None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:13.599596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:13.600290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:13.600572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:13.600843 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] PATH_INFO: `/v3/projects/09ce7d67e5fd43f2a59e314104937d28` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:13.601010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1147746-7f96-4cb8-97a9-1d67039b5291 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:13.601263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1147746-7f96-4cb8-97a9-1d67039b5291 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:13.601544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1147746-7f96-4cb8-97a9-1d67039b5291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:13.602276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09ce7d67e5fd43f2a59e314104937d28)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:13.604305 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:13.633828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-f1147746-7f96-4cb8-97a9-1d67039b5291 None None] MFA Rules not processed for user `072cedf1a6a24f6fa7d7ed92981344dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:13.634968 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f1147746-7f96-4cb8-97a9-1d67039b5291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:13.656390 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09ce7d67e5fd43f2a59e314104937d28', 'request_id': 'req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:13.661788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09ce7d67e5fd43f2a59e314104937d28', 'request_id': 'req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:13.669716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09ce7d67e5fd43f2a59e314104937d28', 'request_id': 'req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:13.681319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] The token cache is being invalidate because project 09ce7d67e5fd43f2a59e314104937d28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:03:13.681692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:13.682345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:13.684802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1127/2300] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:03:13 2022] DELETE /identity/v3/projects/09ce7d67e5fd43f2a59e314104937d28 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:03:13.694085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1174/2301] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:15.032148 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:15.033637 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:15.098033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:15.128829 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2eb1-9f0d-40ec-8db2-e5a77ac49169 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:15.133781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535116f1-63a6-4629-9670-45431ebe4682 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:15.134377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535116f1-63a6-4629-9670-45431ebe4682 service placement] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:15.134651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535116f1-63a6-4629-9670-45431ebe4682 service placement] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:15.134917 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535116f1-63a6-4629-9670-45431ebe4682 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:15.136659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-535116f1-63a6-4629-9670-45431ebe4682 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:15.148872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535116f1-63a6-4629-9670-45431ebe4682 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:15.150723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535116f1-63a6-4629-9670-45431ebe4682 service placement] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:15.162388 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1128/2302] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:22.089006 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1147746-7f96-4cb8-97a9-1d67039b5291 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:22.090603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f1147746-7f96-4cb8-97a9-1d67039b5291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:22.104437 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1147746-7f96-4cb8-97a9-1d67039b5291 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:22.162290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:22.162922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:22.163200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:22.163471 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] PATH_INFO: `/v3/users/50cb67eec9d9445e87fb62bf0fadb907` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:22.164646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50cb67eec9d9445e87fb62bf0fadb907)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:22.166587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:22.216604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50cb67eec9d9445e87fb62bf0fadb907'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:22.217261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50cb67eec9d9445e87fb62bf0fadb907'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:22.224466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50cb67eec9d9445e87fb62bf0fadb907'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:22.225075 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50cb67eec9d9445e87fb62bf0fadb907'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:22.237606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:22.239017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50cb67eec9d9445e87fb62bf0fadb907'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:22.253048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1175/2303] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:03:22 2022] DELETE /identity/v3/users/50cb67eec9d9445e87fb62bf0fadb907 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:22.257644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee90c0-cde8-4e71-b824-a1f11555f1ec None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:22.258252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee90c0-cde8-4e71-b824-a1f11555f1ec None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:22.258252 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee90c0-cde8-4e71-b824-a1f11555f1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:22.279753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-80ee90c0-cde8-4e71-b824-a1f11555f1ec None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:22.281010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-80ee90c0-cde8-4e71-b824-a1f11555f1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:22.344231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1129/2304] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:03:22.354497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:22.356545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:22.443801 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776638c-1489-4afb-9483-a9b809637673 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:22.447582 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ab25d-097b-42e7-bef2-38b2eb386f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:22.448223 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ab25d-097b-42e7-bef2-38b2eb386f0b service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:22.448500 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ab25d-097b-42e7-bef2-38b2eb386f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:22.448766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ab25d-097b-42e7-bef2-38b2eb386f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:22.458013 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ab25d-097b-42e7-bef2-38b2eb386f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:22.459451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ab25d-097b-42e7-bef2-38b2eb386f0b service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:22.469810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1176/2305] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:22.777452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee90c0-cde8-4e71-b824-a1f11555f1ec None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:22.779840 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-80ee90c0-cde8-4e71-b824-a1f11555f1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:22.805712 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee90c0-cde8-4e71-b824-a1f11555f1ec None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:22.811493 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:22.812181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:22.812456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:22.812769 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] PATH_INFO: `/v3/projects/70f7134104124f489332a919cb2a5a3f` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:22.814696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f7134104124f489332a919cb2a5a3f)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:22.815932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:22.870739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f7134104124f489332a919cb2a5a3f', 'request_id': 'req-8e85ce3a-b75f-4023-959c-9e0819943c32'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:22.876817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f7134104124f489332a919cb2a5a3f', 'request_id': 'req-8e85ce3a-b75f-4023-959c-9e0819943c32'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:22.887345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f7134104124f489332a919cb2a5a3f', 'request_id': 'req-8e85ce3a-b75f-4023-959c-9e0819943c32'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:22.901166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] The token cache is being invalidate because project 70f7134104124f489332a919cb2a5a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:03:22.901473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:22.903586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:22.907085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1130/2306] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:03:22 2022] DELETE /identity/v3/projects/70f7134104124f489332a919cb2a5a3f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:22.908883 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ab25d-097b-42e7-bef2-38b2eb386f0b service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:22.919820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ab25d-097b-42e7-bef2-38b2eb386f0b service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:22.987914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:22.988804 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:22.989080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:22.989351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] PATH_INFO: `/v3/users/836b50398d364da3b1cc1cd82bcd3fa2` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:22.990588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=836b50398d364da3b1cc1cd82bcd3fa2)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:22.992572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:23.046343 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '836b50398d364da3b1cc1cd82bcd3fa2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:23.047088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '836b50398d364da3b1cc1cd82bcd3fa2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:23.057254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '836b50398d364da3b1cc1cd82bcd3fa2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:23.057572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '836b50398d364da3b1cc1cd82bcd3fa2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:23.072290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-bac316e9-572c-4508-9159-deaff1bdcfaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '836b50398d364da3b1cc1cd82bcd3fa2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:23.073940 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:INSERT=1,keystone:DELETE=10 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:23.089229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1177/2307] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:03:22 2022] DELETE /identity/v3/users/836b50398d364da3b1cc1cd82bcd3fa2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:23.390146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:23.391826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:23.410859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85ce3a-b75f-4023-959c-9e0819943c32 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:23.461043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:23.473957 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:23.474597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:23.474872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:23.475142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] PATH_INFO: `/v3/projects/885fa5cd06a44c07b234833bfa548e4c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:23.476301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=885fa5cd06a44c07b234833bfa548e4c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:23.478237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:23.527073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885fa5cd06a44c07b234833bfa548e4c', 'request_id': 'req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:23.533389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885fa5cd06a44c07b234833bfa548e4c', 'request_id': 'req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:23.540473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885fa5cd06a44c07b234833bfa548e4c', 'request_id': 'req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:23.552237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] The token cache is being invalidate because project 885fa5cd06a44c07b234833bfa548e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:03:23.552521 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:23.553174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-9fa90814-f088-4d1b-a78b-d0fa6ba97c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:23.554969 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1131/2308] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:03:23 2022] DELETE /identity/v3/projects/885fa5cd06a44c07b234833bfa548e4c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:23.559844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495333f0-439a-49a9-87ca-8969e38e5140 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:23.560156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495333f0-439a-49a9-87ca-8969e38e5140 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:23.560425 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495333f0-439a-49a9-87ca-8969e38e5140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:23.611790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-495333f0-439a-49a9-87ca-8969e38e5140 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:23.613031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-495333f0-439a-49a9-87ca-8969e38e5140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:23.677822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1178/2309] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:23.686285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7f1bc-4876-4f24-b2ce-2ac86c007268 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:23.686563 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7f1bc-4876-4f24-b2ce-2ac86c007268 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:23.686856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7f1bc-4876-4f24-b2ce-2ac86c007268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:23.707132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-9ab7f1bc-4876-4f24-b2ce-2ac86c007268 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:23.708386 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9ab7f1bc-4876-4f24-b2ce-2ac86c007268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:23.716995 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1132/2310] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:03:23.721220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495333f0-439a-49a9-87ca-8969e38e5140 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:23.730727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495333f0-439a-49a9-87ca-8969e38e5140 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:23.735911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabfcfe-8a32-4a37-9440-b61612791eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:23.736527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabfcfe-8a32-4a37-9440-b61612791eca admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:23.736801 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabfcfe-8a32-4a37-9440-b61612791eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:23.737070 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabfcfe-8a32-4a37-9440-b61612791eca admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:23.737715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabfcfe-8a32-4a37-9440-b61612791eca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:03:23.738014 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabfcfe-8a32-4a37-9440-b61612791eca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:23.739616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabfcfe-8a32-4a37-9440-b61612791eca admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:23.751465 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1179/2311] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:03:23.755798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7f1bc-4876-4f24-b2ce-2ac86c007268 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:23.765386 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7f1bc-4876-4f24-b2ce-2ac86c007268 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:23.770187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3268f7-6be1-4858-910c-acfae282ec32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:23.770956 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3268f7-6be1-4858-910c-acfae282ec32 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:23.771259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3268f7-6be1-4858-910c-acfae282ec32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:23.771623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3268f7-6be1-4858-910c-acfae282ec32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:23.772431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3268f7-6be1-4858-910c-acfae282ec32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:23.774891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3268f7-6be1-4858-910c-acfae282ec32 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:23.815913 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1133/2312] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:03:23 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:23.817740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabfcfe-8a32-4a37-9440-b61612791eca admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:23.833548 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabfcfe-8a32-4a37-9440-b61612791eca admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:23.890754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a875e39-0737-47b3-986a-e31aa264df2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:23.891360 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a875e39-0737-47b3-986a-e31aa264df2c admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:23.891632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a875e39-0737-47b3-986a-e31aa264df2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:23.891922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a875e39-0737-47b3-986a-e31aa264df2c admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:23.892561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a875e39-0737-47b3-986a-e31aa264df2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:23.894482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a875e39-0737-47b3-986a-e31aa264df2c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:23.918232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1180/2313] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:03:23 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:23.922949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3268f7-6be1-4858-910c-acfae282ec32 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:23.933511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3268f7-6be1-4858-910c-acfae282ec32 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:23.938090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b2692-4d4b-4136-b82b-92b8c38c7e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:23.938680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b2692-4d4b-4136-b82b-92b8c38c7e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:23.938953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b2692-4d4b-4136-b82b-92b8c38c7e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:23.939220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b2692-4d4b-4136-b82b-92b8c38c7e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:23.939852 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b2692-4d4b-4136-b82b-92b8c38c7e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:23.941461 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b2692-4d4b-4136-b82b-92b8c38c7e67 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:23.951800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1134/2314] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:23.956031 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a875e39-0737-47b3-986a-e31aa264df2c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:23.966893 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a875e39-0737-47b3-986a-e31aa264df2c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:23.971559 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe3327-3cc8-4493-869b-1170ef7dbba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:23.972273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe3327-3cc8-4493-869b-1170ef7dbba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:23.972535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe3327-3cc8-4493-869b-1170ef7dbba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:23.972803 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe3327-3cc8-4493-869b-1170ef7dbba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:23.973446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe3327-3cc8-4493-869b-1170ef7dbba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:23.975043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe3327-3cc8-4493-869b-1170ef7dbba4 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:23.986063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1181/2315] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:23.990353 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b2692-4d4b-4136-b82b-92b8c38c7e67 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:24.000638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b2692-4d4b-4136-b82b-92b8c38c7e67 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:24.005423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621bb4a-9746-4b4c-969c-199da8dfaaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:24.006008 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621bb4a-9746-4b4c-969c-199da8dfaaf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:24.006310 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621bb4a-9746-4b4c-969c-199da8dfaaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:24.006583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621bb4a-9746-4b4c-969c-199da8dfaaf5 admin admin] PATH_INFO: `/v3/projects/b1fe371f6ed8486a8ce3164c623a2ca7/users/99e7fb28ab3d49139503e6b920d2860e/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:24.021805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621bb4a-9746-4b4c-969c-199da8dfaaf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1fe371f6ed8486a8ce3164c623a2ca7, user_id=99e7fb28ab3d49139503e6b920d2860e, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:24.024517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621bb4a-9746-4b4c-969c-199da8dfaaf5 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:24.038116 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1135/2316] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:03:23 2022] PUT /identity/v3/projects/b1fe371f6ed8486a8ce3164c623a2ca7/users/99e7fb28ab3d49139503e6b920d2860e/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:24.048445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d8882-5266-45b4-aeff-f1761ad8818a None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:24.048727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d8882-5266-45b4-aeff-f1761ad8818a None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:24.048997 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d8882-5266-45b4-aeff-f1761ad8818a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:24.076961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-5f5d8882-5266-45b4-aeff-f1761ad8818a None None] MFA Rules not processed for user `99e7fb28ab3d49139503e6b920d2860e`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:24.078161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-5f5d8882-5266-45b4-aeff-f1761ad8818a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:24.098155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:24.144017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1182/2317] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:24.148880 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621bb4a-9746-4b4c-969c-199da8dfaaf5 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:24.209433 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621bb4a-9746-4b4c-969c-199da8dfaaf5 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:24.214008 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286f9f5-795b-4c4b-8368-884501b7a1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:24.214591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286f9f5-795b-4c4b-8368-884501b7a1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:24.214862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286f9f5-795b-4c4b-8368-884501b7a1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:24.215126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286f9f5-795b-4c4b-8368-884501b7a1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:24.215771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286f9f5-795b-4c4b-8368-884501b7a1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:24.217823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286f9f5-795b-4c4b-8368-884501b7a1d3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:24.243584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1136/2318] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:03:24 2022] POST /identity/v3/projects => generated 361 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:03:24.248561 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d8882-5266-45b4-aeff-f1761ad8818a None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:24.259224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d8882-5266-45b4-aeff-f1761ad8818a None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:24.316091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ac6de-71ee-41f6-8161-7dbe9b8a83e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:24.316692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ac6de-71ee-41f6-8161-7dbe9b8a83e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:24.316971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ac6de-71ee-41f6-8161-7dbe9b8a83e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:24.317242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ac6de-71ee-41f6-8161-7dbe9b8a83e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:24.317862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ac6de-71ee-41f6-8161-7dbe9b8a83e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:24.319817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ac6de-71ee-41f6-8161-7dbe9b8a83e6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:24.341544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1183/2319] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:03:24 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:24.345574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286f9f5-795b-4c4b-8368-884501b7a1d3 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:24.351056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:24.357983 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286f9f5-795b-4c4b-8368-884501b7a1d3 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:24.363929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3441fa-f761-46ec-b5b6-e5cda05a3da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:24.364532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3441fa-f761-46ec-b5b6-e5cda05a3da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:24.364811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3441fa-f761-46ec-b5b6-e5cda05a3da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:24.365111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3441fa-f761-46ec-b5b6-e5cda05a3da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:24.365750 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3441fa-f761-46ec-b5b6-e5cda05a3da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:24.367346 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3441fa-f761-46ec-b5b6-e5cda05a3da2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:24.379636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1137/2320] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:24.383854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ac6de-71ee-41f6-8161-7dbe9b8a83e6 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:24.394569 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ac6de-71ee-41f6-8161-7dbe9b8a83e6 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:24.399225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfad09c-d551-4f8d-8d5c-965337d88cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:24.399871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfad09c-d551-4f8d-8d5c-965337d88cea admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:24.400171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfad09c-d551-4f8d-8d5c-965337d88cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:24.400442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfad09c-d551-4f8d-8d5c-965337d88cea admin admin] PATH_INFO: `/v3/projects/4d086b5d9ada49cfbbc4a9c79f552e33/users/2eaed00975ef49fb85aaa2182d6986cb/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:24.415857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfad09c-d551-4f8d-8d5c-965337d88cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d086b5d9ada49cfbbc4a9c79f552e33, user_id=2eaed00975ef49fb85aaa2182d6986cb, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:24.419267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfad09c-d551-4f8d-8d5c-965337d88cea admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:24.432216 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1184/2321] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:03:24 2022] PUT /identity/v3/projects/4d086b5d9ada49cfbbc4a9c79f552e33/users/2eaed00975ef49fb85aaa2182d6986cb/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:24.436242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3441fa-f761-46ec-b5b6-e5cda05a3da2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:24.446620 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3441fa-f761-46ec-b5b6-e5cda05a3da2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:24.503210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d0ca0-02cb-4862-8d3b-eeae4a2791be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:24.503811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d0ca0-02cb-4862-8d3b-eeae4a2791be admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:24.504120 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d0ca0-02cb-4862-8d3b-eeae4a2791be admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:24.504444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d0ca0-02cb-4862-8d3b-eeae4a2791be admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:24.505042 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d0ca0-02cb-4862-8d3b-eeae4a2791be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:24.506630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d0ca0-02cb-4862-8d3b-eeae4a2791be admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:24.516515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1138/2322] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:24.520764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfad09c-d551-4f8d-8d5c-965337d88cea admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:24.531333 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfad09c-d551-4f8d-8d5c-965337d88cea admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:24.535978 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425b686-a101-4007-83b5-bfd77f3f897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:24.536607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425b686-a101-4007-83b5-bfd77f3f897c admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:24.536882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425b686-a101-4007-83b5-bfd77f3f897c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:24.537149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425b686-a101-4007-83b5-bfd77f3f897c admin admin] PATH_INFO: `/v3/projects/4d086b5d9ada49cfbbc4a9c79f552e33/users/2eaed00975ef49fb85aaa2182d6986cb/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:24.539056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425b686-a101-4007-83b5-bfd77f3f897c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d086b5d9ada49cfbbc4a9c79f552e33, user_id=2eaed00975ef49fb85aaa2182d6986cb, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:24.541715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425b686-a101-4007-83b5-bfd77f3f897c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:24.552238 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1185/2323] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:03:24 2022] PUT /identity/v3/projects/4d086b5d9ada49cfbbc4a9c79f552e33/users/2eaed00975ef49fb85aaa2182d6986cb/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:24.562582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677aa9-4d70-4ac9-92fc-f0fd94826053 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:24.562856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677aa9-4d70-4ac9-92fc-f0fd94826053 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:24.563121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677aa9-4d70-4ac9-92fc-f0fd94826053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:24.591534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-83677aa9-4d70-4ac9-92fc-f0fd94826053 None None] MFA Rules not processed for user `2eaed00975ef49fb85aaa2182d6986cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:24.592808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-83677aa9-4d70-4ac9-92fc-f0fd94826053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:24.660874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1139/2324] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:24 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:24.672746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425b686-a101-4007-83b5-bfd77f3f897c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:24.674498 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8425b686-a101-4007-83b5-bfd77f3f897c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:24.732812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425b686-a101-4007-83b5-bfd77f3f897c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:24.736446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833825b7-b93b-41ca-8d08-262d78855da4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:24.737040 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833825b7-b93b-41ca-8d08-262d78855da4 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:24.737311 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833825b7-b93b-41ca-8d08-262d78855da4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:24.737578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833825b7-b93b-41ca-8d08-262d78855da4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:24.746420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833825b7-b93b-41ca-8d08-262d78855da4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:24.747918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833825b7-b93b-41ca-8d08-262d78855da4 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:24.757138 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1186/2325] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:24 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:25.192106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83677aa9-4d70-4ac9-92fc-f0fd94826053 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:25.255121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83677aa9-4d70-4ac9-92fc-f0fd94826053 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:25.259710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:25.260480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:25.260758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:25.261049 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] PATH_INFO: `/v3/users/99e7fb28ab3d49139503e6b920d2860e` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:25.262219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e7fb28ab3d49139503e6b920d2860e)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:25.265012 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:25.278196 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:25.310641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e7fb28ab3d49139503e6b920d2860e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:25.311199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e7fb28ab3d49139503e6b920d2860e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:25.318031 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e7fb28ab3d49139503e6b920d2860e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:25.318646 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e7fb28ab3d49139503e6b920d2860e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:25.330363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e7fb28ab3d49139503e6b920d2860e'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:25.343352 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1140/2326] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:03:25 2022] DELETE /identity/v3/users/99e7fb28ab3d49139503e6b920d2860e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:25.347841 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e9dba-40a4-4a05-917a-6e00fc5953b8 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:25.348372 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e9dba-40a4-4a05-917a-6e00fc5953b8 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:25.348482 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e9dba-40a4-4a05-917a-6e00fc5953b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:25.367505 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-cb1e9dba-40a4-4a05-917a-6e00fc5953b8 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:25.368712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cb1e9dba-40a4-4a05-917a-6e00fc5953b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:25.380164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:25.431256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1187/2327] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:25.440820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:25.442217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:25.498722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56771da9-4664-455c-a7f7-b2bd750d3ff6 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:25.502328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc14de8-b2dc-4c25-a4d8-dfbd4207911a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:25.502920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc14de8-b2dc-4c25-a4d8-dfbd4207911a service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:25.503195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc14de8-b2dc-4c25-a4d8-dfbd4207911a service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:25.503465 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc14de8-b2dc-4c25-a4d8-dfbd4207911a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:25.518907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc14de8-b2dc-4c25-a4d8-dfbd4207911a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:25.520370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc14de8-b2dc-4c25-a4d8-dfbd4207911a service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:25.530523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1141/2328] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:25.805666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e9dba-40a4-4a05-917a-6e00fc5953b8 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:25.807245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-cb1e9dba-40a4-4a05-917a-6e00fc5953b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:25.821142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e9dba-40a4-4a05-917a-6e00fc5953b8 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:25.826573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:25.827735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:25.828072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:25.828522 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] PATH_INFO: `/v3/projects/b1fe371f6ed8486a8ce3164c623a2ca7` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:25.829666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1fe371f6ed8486a8ce3164c623a2ca7)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:25.831583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:25.878588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1fe371f6ed8486a8ce3164c623a2ca7', 'request_id': 'req-b7e95732-2ea6-4be7-b658-79651d478f42'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:25.883856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1fe371f6ed8486a8ce3164c623a2ca7', 'request_id': 'req-b7e95732-2ea6-4be7-b658-79651d478f42'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:25.891736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1fe371f6ed8486a8ce3164c623a2ca7', 'request_id': 'req-b7e95732-2ea6-4be7-b658-79651d478f42'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:25.902688 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] The token cache is being invalidate because project b1fe371f6ed8486a8ce3164c623a2ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:03:25.902971 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:25.903902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:25.905370 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1188/2329] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:03:25 2022] DELETE /identity/v3/projects/b1fe371f6ed8486a8ce3164c623a2ca7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:25.909728 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc14de8-b2dc-4c25-a4d8-dfbd4207911a service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:25.919308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc14de8-b2dc-4c25-a4d8-dfbd4207911a service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:25.975179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:25.975792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:25.976095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:25.976417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] PATH_INFO: `/v3/users/2eaed00975ef49fb85aaa2182d6986cb` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:25.977512 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eaed00975ef49fb85aaa2182d6986cb)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:25.979405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:26.017726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:26.027809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eaed00975ef49fb85aaa2182d6986cb'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:26.029268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eaed00975ef49fb85aaa2182d6986cb'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:26.036445 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eaed00975ef49fb85aaa2182d6986cb'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:26.037072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eaed00975ef49fb85aaa2182d6986cb'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:26.049519 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-04188030-f982-49fb-89cf-c0377bf6b7fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eaed00975ef49fb85aaa2182d6986cb'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:26.063014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1142/2330] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:03:25 2022] DELETE /identity/v3/users/2eaed00975ef49fb85aaa2182d6986cb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:26.335072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:26.336551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:26.354473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e95732-2ea6-4be7-b658-79651d478f42 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:26.416856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:26.417411 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:26.417687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:26.417958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] PATH_INFO: `/v3/projects/4d086b5d9ada49cfbbc4a9c79f552e33` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:26.419581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d086b5d9ada49cfbbc4a9c79f552e33)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:26.421607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:26.463479 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:26.467687 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d086b5d9ada49cfbbc4a9c79f552e33', 'request_id': 'req-d0edaedb-0657-488a-ac1b-04ce921b94b8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:26.474489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d086b5d9ada49cfbbc4a9c79f552e33', 'request_id': 'req-d0edaedb-0657-488a-ac1b-04ce921b94b8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:26.482181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d086b5d9ada49cfbbc4a9c79f552e33', 'request_id': 'req-d0edaedb-0657-488a-ac1b-04ce921b94b8'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:26.493091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] The token cache is being invalidate because project 4d086b5d9ada49cfbbc4a9c79f552e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:03:26.493378 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:26.494007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d0edaedb-0657-488a-ac1b-04ce921b94b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:26.495474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1189/2331] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:03:26 2022] DELETE /identity/v3/projects/4d086b5d9ada49cfbbc4a9c79f552e33 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:26.501027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50a24-3d28-498c-822a-bbd8adfe82ae None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:26.501525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50a24-3d28-498c-822a-bbd8adfe82ae None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:26.501675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50a24-3d28-498c-822a-bbd8adfe82ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:26.520795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-d3e50a24-3d28-498c-822a-bbd8adfe82ae None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:26.521994 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d3e50a24-3d28-498c-822a-bbd8adfe82ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:26.580426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1143/2332] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:26.589313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ba26-7b8c-41bb-bbfd-0a2a0ee445af None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:26.589622 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ba26-7b8c-41bb-bbfd-0a2a0ee445af None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:26.589899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ba26-7b8c-41bb-bbfd-0a2a0ee445af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:26.610419 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-c006ba26-7b8c-41bb-bbfd-0a2a0ee445af None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:26.611622 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c006ba26-7b8c-41bb-bbfd-0a2a0ee445af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:26.619737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1190/2333] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:26.624377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e50a24-3d28-498c-822a-bbd8adfe82ae None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:26.633929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e50a24-3d28-498c-822a-bbd8adfe82ae None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:26.638553 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaa30a-a909-4003-a7d8-a039804e7c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:26.639150 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaa30a-a909-4003-a7d8-a039804e7c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:26.639429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaa30a-a909-4003-a7d8-a039804e7c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:26.639700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaa30a-a909-4003-a7d8-a039804e7c4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:26.640376 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaa30a-a909-4003-a7d8-a039804e7c4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:03:26.640678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaa30a-a909-4003-a7d8-a039804e7c4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:26.642274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaa30a-a909-4003-a7d8-a039804e7c4b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:26.654260 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1144/2334] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 18:03:26.658385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006ba26-7b8c-41bb-bbfd-0a2a0ee445af None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:26.667644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006ba26-7b8c-41bb-bbfd-0a2a0ee445af None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:26.673056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c98c9-da3a-4b40-bd38-5ada5d552a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:26.673648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c98c9-da3a-4b40-bd38-5ada5d552a03 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:26.673924 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c98c9-da3a-4b40-bd38-5ada5d552a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:26.674192 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c98c9-da3a-4b40-bd38-5ada5d552a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:26.674818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c98c9-da3a-4b40-bd38-5ada5d552a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:26.676752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c98c9-da3a-4b40-bd38-5ada5d552a03 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:26.704034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1191/2335] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:03:26 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:26.707832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaa30a-a909-4003-a7d8-a039804e7c4b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:26.718720 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaa30a-a909-4003-a7d8-a039804e7c4b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:26.775678 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01ff7-5f08-42bf-a010-51a1ff00f820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:26.776405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01ff7-5f08-42bf-a010-51a1ff00f820 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:26.776684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01ff7-5f08-42bf-a010-51a1ff00f820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:26.776954 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01ff7-5f08-42bf-a010-51a1ff00f820 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:26.777582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c01ff7-5f08-42bf-a010-51a1ff00f820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:26.780087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c01ff7-5f08-42bf-a010-51a1ff00f820 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:26.801578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1145/2336] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:03:26 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:26.805903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c98c9-da3a-4b40-bd38-5ada5d552a03 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:26.816829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c98c9-da3a-4b40-bd38-5ada5d552a03 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:26.821406 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5d057-5a5e-4e9e-8e2b-fabbaf661bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:26.821994 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5d057-5a5e-4e9e-8e2b-fabbaf661bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:26.822267 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5d057-5a5e-4e9e-8e2b-fabbaf661bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:26.822533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5d057-5a5e-4e9e-8e2b-fabbaf661bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:26.823169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5d057-5a5e-4e9e-8e2b-fabbaf661bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:26.824801 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5d057-5a5e-4e9e-8e2b-fabbaf661bb8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:26.834606 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1192/2337] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:26.838325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01ff7-5f08-42bf-a010-51a1ff00f820 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:26.848839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01ff7-5f08-42bf-a010-51a1ff00f820 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:26.853623 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab41ab2-3552-4c66-8478-34f528d07463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:26.854208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab41ab2-3552-4c66-8478-34f528d07463 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:26.854505 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab41ab2-3552-4c66-8478-34f528d07463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:26.854770 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab41ab2-3552-4c66-8478-34f528d07463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:26.855397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab41ab2-3552-4c66-8478-34f528d07463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:26.857487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab41ab2-3552-4c66-8478-34f528d07463 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:26.867280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1146/2338] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:26.871168 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5d057-5a5e-4e9e-8e2b-fabbaf661bb8 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:26.881619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5d057-5a5e-4e9e-8e2b-fabbaf661bb8 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:26.886188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d1ace-fd1f-4d29-a190-a4e081ac3178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:26.886820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d1ace-fd1f-4d29-a190-a4e081ac3178 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:26.887096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d1ace-fd1f-4d29-a190-a4e081ac3178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:26.887364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d1ace-fd1f-4d29-a190-a4e081ac3178 admin admin] PATH_INFO: `/v3/projects/f50e6ead4b964bfaa8fdee00da4c2bc0/users/fa123704273f4937aa4b34a1b7b08145/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:26.902080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d1ace-fd1f-4d29-a190-a4e081ac3178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50e6ead4b964bfaa8fdee00da4c2bc0, user_id=fa123704273f4937aa4b34a1b7b08145, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:26.905929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d1ace-fd1f-4d29-a190-a4e081ac3178 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:26.918694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1193/2339] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:03:26 2022] PUT /identity/v3/projects/f50e6ead4b964bfaa8fdee00da4c2bc0/users/fa123704273f4937aa4b34a1b7b08145/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:26.929835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fae33-89c8-40fb-a1d3-f076347ad85d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:26.930158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fae33-89c8-40fb-a1d3-f076347ad85d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:26.930408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fae33-89c8-40fb-a1d3-f076347ad85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:26.958090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-366fae33-89c8-40fb-a1d3-f076347ad85d None None] MFA Rules not processed for user `fa123704273f4937aa4b34a1b7b08145`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:26.959270 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-366fae33-89c8-40fb-a1d3-f076347ad85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:26.967821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:27.027034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1147/2340] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:27.039748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d1ace-fd1f-4d29-a190-a4e081ac3178 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:27.041618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9c0d1ace-fd1f-4d29-a190-a4e081ac3178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:27.119705 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d1ace-fd1f-4d29-a190-a4e081ac3178 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:27.123550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2342bb-5922-4864-9115-732af8707781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:27.124179 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2342bb-5922-4864-9115-732af8707781 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:27.124457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2342bb-5922-4864-9115-732af8707781 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:27.124727 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2342bb-5922-4864-9115-732af8707781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:27.132982 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2342bb-5922-4864-9115-732af8707781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:27.134418 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2342bb-5922-4864-9115-732af8707781 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:27.143557 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1194/2341] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:27.163777 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fae33-89c8-40fb-a1d3-f076347ad85d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:27.225747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fae33-89c8-40fb-a1d3-f076347ad85d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:27.230230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:27.230856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:27.231133 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:27.231467 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] PATH_INFO: `/v3/users/fa123704273f4937aa4b34a1b7b08145` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:27.232814 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa123704273f4937aa4b34a1b7b08145)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:27.234728 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:27.280061 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa123704273f4937aa4b34a1b7b08145'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:27.280699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa123704273f4937aa4b34a1b7b08145'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:27.287621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa123704273f4937aa4b34a1b7b08145'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:27.288294 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa123704273f4937aa4b34a1b7b08145'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:27.300174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa123704273f4937aa4b34a1b7b08145'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:27.314192 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1148/2342] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:03:27 2022] DELETE /identity/v3/users/fa123704273f4937aa4b34a1b7b08145 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:27.318846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5a9d9-2846-4d85-a59b-32425007cfa7 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:27.319126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5a9d9-2846-4d85-a59b-32425007cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:27.319394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5a9d9-2846-4d85-a59b-32425007cfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:27.338638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-81d5a9d9-2846-4d85-a59b-32425007cfa7 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:27.339832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-81d5a9d9-2846-4d85-a59b-32425007cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:27.387347 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:27.405340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1195/2343] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:27.415208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:27.416768 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:27.452087 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:27.479422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d6c22-5558-4e30-b83a-ad8d3cfb5e9b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:27.483221 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97f50b-912b-4575-b661-037e9c0d230a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:27.483830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97f50b-912b-4575-b661-037e9c0d230a service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:27.484143 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97f50b-912b-4575-b661-037e9c0d230a service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:27.484420 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97f50b-912b-4575-b661-037e9c0d230a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:27.493403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97f50b-912b-4575-b661-037e9c0d230a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:27.494830 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97f50b-912b-4575-b661-037e9c0d230a service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:27.505030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1149/2344] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:27.765635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5a9d9-2846-4d85-a59b-32425007cfa7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:27.767155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-81d5a9d9-2846-4d85-a59b-32425007cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:27.785036 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5a9d9-2846-4d85-a59b-32425007cfa7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:27.790567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:27.791260 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:27.791540 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:27.791801 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] PATH_INFO: `/v3/projects/f50e6ead4b964bfaa8fdee00da4c2bc0` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:27.792946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50e6ead4b964bfaa8fdee00da4c2bc0)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:27.795400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:27.842323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50e6ead4b964bfaa8fdee00da4c2bc0', 'request_id': 'req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:27.847753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50e6ead4b964bfaa8fdee00da4c2bc0', 'request_id': 'req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:27.855128 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50e6ead4b964bfaa8fdee00da4c2bc0', 'request_id': 'req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:27.866996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] The token cache is being invalidate because project f50e6ead4b964bfaa8fdee00da4c2bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:03:27.867280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:27.867989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-0201d0ba-f1cd-433c-ac97-1e3ae9bbc98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:27.869478 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1196/2345] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:03:27 2022] DELETE /identity/v3/projects/f50e6ead4b964bfaa8fdee00da4c2bc0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:27.875057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e1202-6cce-4d3d-b151-75f9a5d9480d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:27.875340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e1202-6cce-4d3d-b151-75f9a5d9480d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:27.875613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e1202-6cce-4d3d-b151-75f9a5d9480d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:27.894698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-995e1202-6cce-4d3d-b151-75f9a5d9480d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:27.895918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-995e1202-6cce-4d3d-b151-75f9a5d9480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:27.954045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1150/2346] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:27.961952 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20f71f-f0c8-478c-a6f4-a2c59c438182 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:27.962262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20f71f-f0c8-478c-a6f4-a2c59c438182 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:27.962550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20f71f-f0c8-478c-a6f4-a2c59c438182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:27.982545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ee20f71f-f0c8-478c-a6f4-a2c59c438182 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:27.983737 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ee20f71f-f0c8-478c-a6f4-a2c59c438182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:27.991733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1197/2347] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:27.995824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e1202-6cce-4d3d-b151-75f9a5d9480d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:28.005389 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e1202-6cce-4d3d-b151-75f9a5d9480d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:28.009861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fa5b2-0a9b-4df7-a235-c0db631e3380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:28.010393 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fa5b2-0a9b-4df7-a235-c0db631e3380 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.010664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fa5b2-0a9b-4df7-a235-c0db631e3380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.010927 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fa5b2-0a9b-4df7-a235-c0db631e3380 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.011552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fa5b2-0a9b-4df7-a235-c0db631e3380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:03:28.011864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fa5b2-0a9b-4df7-a235-c0db631e3380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:28.013953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fa5b2-0a9b-4df7-a235-c0db631e3380 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:28.025117 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1151/2348] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:03:28.030000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20f71f-f0c8-478c-a6f4-a2c59c438182 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:28.039088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20f71f-f0c8-478c-a6f4-a2c59c438182 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:28.043749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f34b0d-780b-4045-8aa1-8c4d3a930229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:28.044337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f34b0d-780b-4045-8aa1-8c4d3a930229 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.044612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f34b0d-780b-4045-8aa1-8c4d3a930229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.044880 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f34b0d-780b-4045-8aa1-8c4d3a930229 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.045506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f34b0d-780b-4045-8aa1-8c4d3a930229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:28.047402 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f34b0d-780b-4045-8aa1-8c4d3a930229 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:28.076212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1198/2349] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:03:28 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:28.080335 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fa5b2-0a9b-4df7-a235-c0db631e3380 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:28.090307 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fa5b2-0a9b-4df7-a235-c0db631e3380 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:28.149727 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e594db2-1746-4174-a897-6c20fa6385bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:28.150476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594db2-1746-4174-a897-6c20fa6385bb admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.150848 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594db2-1746-4174-a897-6c20fa6385bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.151220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594db2-1746-4174-a897-6c20fa6385bb admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.151971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e594db2-1746-4174-a897-6c20fa6385bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:28.154185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e594db2-1746-4174-a897-6c20fa6385bb admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:28.178511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1152/2350] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:03:28 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:03:28.180258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f34b0d-780b-4045-8aa1-8c4d3a930229 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:28.191818 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f34b0d-780b-4045-8aa1-8c4d3a930229 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:28.196765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07801758-6737-4578-9817-bc9966f533bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:28.197558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07801758-6737-4578-9817-bc9966f533bd admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.197827 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07801758-6737-4578-9817-bc9966f533bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.198092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07801758-6737-4578-9817-bc9966f533bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.198731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07801758-6737-4578-9817-bc9966f533bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:28.201080 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07801758-6737-4578-9817-bc9966f533bd admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:28.211545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1199/2351] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:28.215716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e594db2-1746-4174-a897-6c20fa6385bb admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:28.227520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e594db2-1746-4174-a897-6c20fa6385bb admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:28.232615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92a2d9-5f6d-401d-9046-1754656d6331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:28.233215 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92a2d9-5f6d-401d-9046-1754656d6331 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.233516 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92a2d9-5f6d-401d-9046-1754656d6331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.233787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92a2d9-5f6d-401d-9046-1754656d6331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.234435 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92a2d9-5f6d-401d-9046-1754656d6331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:28.236057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92a2d9-5f6d-401d-9046-1754656d6331 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:28.247343 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1153/2352] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:28.251599 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07801758-6737-4578-9817-bc9966f533bd admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:28.261853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07801758-6737-4578-9817-bc9966f533bd admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:28.267302 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57980a4b-110a-4128-b503-e38c7f70c4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:28.268142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980a4b-110a-4128-b503-e38c7f70c4b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.268485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980a4b-110a-4128-b503-e38c7f70c4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.268819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980a4b-110a-4128-b503-e38c7f70c4b6 admin admin] PATH_INFO: `/v3/projects/3f8922d728494a18b085d34316725fad/users/a07fce24dd6740f6be4efe51ae3574ba/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.284523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57980a4b-110a-4128-b503-e38c7f70c4b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f8922d728494a18b085d34316725fad, user_id=a07fce24dd6740f6be4efe51ae3574ba, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:28.287290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57980a4b-110a-4128-b503-e38c7f70c4b6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:28.302349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1200/2353] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:03:28 2022] PUT /identity/v3/projects/3f8922d728494a18b085d34316725fad/users/a07fce24dd6740f6be4efe51ae3574ba/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:28.315112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18059ef6-b051-4e62-8cc1-66e484dc97f5 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.315398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18059ef6-b051-4e62-8cc1-66e484dc97f5 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.315668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18059ef6-b051-4e62-8cc1-66e484dc97f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.350324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-18059ef6-b051-4e62-8cc1-66e484dc97f5 None None] MFA Rules not processed for user `a07fce24dd6740f6be4efe51ae3574ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:28.352085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-18059ef6-b051-4e62-8cc1-66e484dc97f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:28.379287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:28.418370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1154/2354] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:28.429869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57980a4b-110a-4128-b503-e38c7f70c4b6 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:28.431626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-57980a4b-110a-4128-b503-e38c7f70c4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:28.489571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57980a4b-110a-4128-b503-e38c7f70c4b6 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:28.493175 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5f56b-d76a-4ecd-8972-18753195ea4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:28.493769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5f56b-d76a-4ecd-8972-18753195ea4e service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.494043 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5f56b-d76a-4ecd-8972-18753195ea4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.494306 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5f56b-d76a-4ecd-8972-18753195ea4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.503067 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5f56b-d76a-4ecd-8972-18753195ea4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:28.504537 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5f56b-d76a-4ecd-8972-18753195ea4e service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:28.513220 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1201/2355] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:28.534906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18059ef6-b051-4e62-8cc1-66e484dc97f5 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:28.602602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18059ef6-b051-4e62-8cc1-66e484dc97f5 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:28.609333 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:28.610115 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.610396 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.610726 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] PATH_INFO: `/v3/users/a07fce24dd6740f6be4efe51ae3574ba` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.611807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07fce24dd6740f6be4efe51ae3574ba)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:28.613775 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:28.681262 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07fce24dd6740f6be4efe51ae3574ba'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:28.681995 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07fce24dd6740f6be4efe51ae3574ba'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:28.691144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07fce24dd6740f6be4efe51ae3574ba'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:28.692144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07fce24dd6740f6be4efe51ae3574ba'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:28.706591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07fce24dd6740f6be4efe51ae3574ba'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:28.722894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1155/2356] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:03:28 2022] DELETE /identity/v3/users/a07fce24dd6740f6be4efe51ae3574ba => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:28.727588 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc0391-43fc-4dc9-97c5-2067aa7e796d None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.728085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc0391-43fc-4dc9-97c5-2067aa7e796d None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.728451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc0391-43fc-4dc9-97c5-2067aa7e796d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.751088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-bedc0391-43fc-4dc9-97c5-2067aa7e796d None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:28.752618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-bedc0391-43fc-4dc9-97c5-2067aa7e796d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:28.762428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:28.829976 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1202/2357] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:28.840345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:28.842721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:28.902893 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:28.911304 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978a69f-c436-4e24-b090-6f0685061370 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:28.916344 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a0d60-d50c-407b-bf06-eebd5bfc0ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:28.917278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a0d60-d50c-407b-bf06-eebd5bfc0ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:28.917622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a0d60-d50c-407b-bf06-eebd5bfc0ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:28.917929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a0d60-d50c-407b-bf06-eebd5bfc0ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:28.928585 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a0d60-d50c-407b-bf06-eebd5bfc0ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:28.930658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a0d60-d50c-407b-bf06-eebd5bfc0ec7 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:28.942529 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1156/2358] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:29.216936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc0391-43fc-4dc9-97c5-2067aa7e796d None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:29.218471 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-bedc0391-43fc-4dc9-97c5-2067aa7e796d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:29.232456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc0391-43fc-4dc9-97c5-2067aa7e796d None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:29.237086 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:29.237712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.237991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.238262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] PATH_INFO: `/v3/projects/3f8922d728494a18b085d34316725fad` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.239377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f8922d728494a18b085d34316725fad)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:29.242053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:29.288962 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f8922d728494a18b085d34316725fad', 'request_id': 'req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:29.295241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f8922d728494a18b085d34316725fad', 'request_id': 'req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:29.302854 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f8922d728494a18b085d34316725fad', 'request_id': 'req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:29.315182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] The token cache is being invalidate because project 3f8922d728494a18b085d34316725fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:03:29.315467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:29.316129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-fe90fd8f-0c07-450b-96d5-c36fd8a0023f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:29.317641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1203/2359] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:03:29 2022] DELETE /identity/v3/projects/3f8922d728494a18b085d34316725fad => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:29.322495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b809c1f-7a0c-4968-a6f9-88ba6ee3c23e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.322803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b809c1f-7a0c-4968-a6f9-88ba6ee3c23e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.323074 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b809c1f-7a0c-4968-a6f9-88ba6ee3c23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.342373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-8b809c1f-7a0c-4968-a6f9-88ba6ee3c23e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:29.343565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8b809c1f-7a0c-4968-a6f9-88ba6ee3c23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:29.402217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1157/2360] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:29.409995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1588951-c79a-4ee7-bdbc-5aae19817f85 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.410303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1588951-c79a-4ee7-bdbc-5aae19817f85 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.410572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1588951-c79a-4ee7-bdbc-5aae19817f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.431525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-a1588951-c79a-4ee7-bdbc-5aae19817f85 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:29.432747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a1588951-c79a-4ee7-bdbc-5aae19817f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:29.441732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1204/2361] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:29.445846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b809c1f-7a0c-4968-a6f9-88ba6ee3c23e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:29.455723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b809c1f-7a0c-4968-a6f9-88ba6ee3c23e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:29.460250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec8a75-a4a9-4c13-8c1d-64d459a82789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:29.460839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec8a75-a4a9-4c13-8c1d-64d459a82789 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.461112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec8a75-a4a9-4c13-8c1d-64d459a82789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.461377 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec8a75-a4a9-4c13-8c1d-64d459a82789 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.462006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec8a75-a4a9-4c13-8c1d-64d459a82789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:03:29.462303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec8a75-a4a9-4c13-8c1d-64d459a82789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:29.463931 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec8a75-a4a9-4c13-8c1d-64d459a82789 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:29.475645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1158/2362] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:03:29.479707 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1588951-c79a-4ee7-bdbc-5aae19817f85 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:29.489392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1588951-c79a-4ee7-bdbc-5aae19817f85 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:29.494023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8196e47-9bad-4dce-8522-c022482799e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:29.494603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8196e47-9bad-4dce-8522-c022482799e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.494881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8196e47-9bad-4dce-8522-c022482799e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.495141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8196e47-9bad-4dce-8522-c022482799e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.495769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8196e47-9bad-4dce-8522-c022482799e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:29.498620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8196e47-9bad-4dce-8522-c022482799e7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:29.525390 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1205/2363] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:03:29 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:03:29.529529 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec8a75-a4a9-4c13-8c1d-64d459a82789 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:29.539267 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec8a75-a4a9-4c13-8c1d-64d459a82789 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:29.595743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce246874-81a9-4d9a-a1f0-4d92d2f5a229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:29.596306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce246874-81a9-4d9a-a1f0-4d92d2f5a229 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.596586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce246874-81a9-4d9a-a1f0-4d92d2f5a229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.596853 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce246874-81a9-4d9a-a1f0-4d92d2f5a229 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.597471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce246874-81a9-4d9a-a1f0-4d92d2f5a229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:29.599369 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce246874-81a9-4d9a-a1f0-4d92d2f5a229 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:29.620059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1159/2364] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:03:29 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:29.624253 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8196e47-9bad-4dce-8522-c022482799e7 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:29.634360 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8196e47-9bad-4dce-8522-c022482799e7 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:29.638961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c14640-fa13-4300-bbe5-024ff7f71131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:29.639547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c14640-fa13-4300-bbe5-024ff7f71131 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.639817 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c14640-fa13-4300-bbe5-024ff7f71131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.640120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c14640-fa13-4300-bbe5-024ff7f71131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.640754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c14640-fa13-4300-bbe5-024ff7f71131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:29.642341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c14640-fa13-4300-bbe5-024ff7f71131 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:29.652126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1206/2365] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:29.656403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce246874-81a9-4d9a-a1f0-4d92d2f5a229 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:29.666186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce246874-81a9-4d9a-a1f0-4d92d2f5a229 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:29.671332 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea801eb-d553-4f65-9809-5ef473d54edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:29.671942 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea801eb-d553-4f65-9809-5ef473d54edc admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.672219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea801eb-d553-4f65-9809-5ef473d54edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.672482 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea801eb-d553-4f65-9809-5ef473d54edc admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.673136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea801eb-d553-4f65-9809-5ef473d54edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:29.674729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea801eb-d553-4f65-9809-5ef473d54edc admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:29.685436 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1160/2366] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:29.689985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c14640-fa13-4300-bbe5-024ff7f71131 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:29.699959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c14640-fa13-4300-bbe5-024ff7f71131 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:29.704650 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d3165-2a4a-4f67-ba61-84b6242ee32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:29.705396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d3165-2a4a-4f67-ba61-84b6242ee32e admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.705678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d3165-2a4a-4f67-ba61-84b6242ee32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.705951 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d3165-2a4a-4f67-ba61-84b6242ee32e admin admin] PATH_INFO: `/v3/projects/60e39841d3994e29b603f8695dbe5b3d/users/41d4a769f828433b9a401a5195a9197b/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.721073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d3165-2a4a-4f67-ba61-84b6242ee32e admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e39841d3994e29b603f8695dbe5b3d, user_id=41d4a769f828433b9a401a5195a9197b, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:29.723743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d3165-2a4a-4f67-ba61-84b6242ee32e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:29.737254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1207/2367] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:03:29 2022] PUT /identity/v3/projects/60e39841d3994e29b603f8695dbe5b3d/users/41d4a769f828433b9a401a5195a9197b/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:29.747358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e899591-ce56-4575-a440-a28d55ee7086 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.747660 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e899591-ce56-4575-a440-a28d55ee7086 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.747959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e899591-ce56-4575-a440-a28d55ee7086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.774693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6e899591-ce56-4575-a440-a28d55ee7086 None None] MFA Rules not processed for user `41d4a769f828433b9a401a5195a9197b`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:29.775860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6e899591-ce56-4575-a440-a28d55ee7086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:29.819269 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:29.841180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1161/2368] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:29.845637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d3165-2a4a-4f67-ba61-84b6242ee32e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:29.879946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:29.908766 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d3165-2a4a-4f67-ba61-84b6242ee32e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:29.913442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1c34e-de4d-495f-a791-f43d7b1e3186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:29.914065 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1c34e-de4d-495f-a791-f43d7b1e3186 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:29.914336 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1c34e-de4d-495f-a791-f43d7b1e3186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:29.914601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1c34e-de4d-495f-a791-f43d7b1e3186 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:29.915255 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1c34e-de4d-495f-a791-f43d7b1e3186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:29.917177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1c34e-de4d-495f-a791-f43d7b1e3186 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:29.943995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1208/2369] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:03:29 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:29.948184 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e899591-ce56-4575-a440-a28d55ee7086 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:29.958975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e899591-ce56-4575-a440-a28d55ee7086 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.014228 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a748d7f-9f6f-409d-9bdf-d169690be802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.014860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748d7f-9f6f-409d-9bdf-d169690be802 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.015135 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748d7f-9f6f-409d-9bdf-d169690be802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.015404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748d7f-9f6f-409d-9bdf-d169690be802 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.016067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a748d7f-9f6f-409d-9bdf-d169690be802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:30.017977 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a748d7f-9f6f-409d-9bdf-d169690be802 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:30.040752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1162/2370] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:03:29 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:30.044505 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1c34e-de4d-495f-a791-f43d7b1e3186 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:30.056774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1c34e-de4d-495f-a791-f43d7b1e3186 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.061982 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8fe0e-7d8b-4436-8478-064fe34a9656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.062612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8fe0e-7d8b-4436-8478-064fe34a9656 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.062884 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8fe0e-7d8b-4436-8478-064fe34a9656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.063147 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8fe0e-7d8b-4436-8478-064fe34a9656 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.063848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8fe0e-7d8b-4436-8478-064fe34a9656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:30.065502 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8fe0e-7d8b-4436-8478-064fe34a9656 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:30.076535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1209/2371] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:30.080684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a748d7f-9f6f-409d-9bdf-d169690be802 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:30.091404 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a748d7f-9f6f-409d-9bdf-d169690be802 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.096429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c410b7d-c26b-467a-a054-9fe6feb9fd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.097059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410b7d-c26b-467a-a054-9fe6feb9fd3e admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.097332 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410b7d-c26b-467a-a054-9fe6feb9fd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.097620 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410b7d-c26b-467a-a054-9fe6feb9fd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.098284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c410b7d-c26b-467a-a054-9fe6feb9fd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:30.099870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c410b7d-c26b-467a-a054-9fe6feb9fd3e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:30.110663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1163/2372] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:30.115041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8fe0e-7d8b-4436-8478-064fe34a9656 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:30.125238 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8fe0e-7d8b-4436-8478-064fe34a9656 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.129851 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552970c6-bd13-47e3-98bc-c57b45dc2a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.130545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552970c6-bd13-47e3-98bc-c57b45dc2a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.131916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552970c6-bd13-47e3-98bc-c57b45dc2a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.131916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552970c6-bd13-47e3-98bc-c57b45dc2a5d admin admin] PATH_INFO: `/v3/projects/acdd3142f6f346e8a1399fe19c8c0ca5/users/7adea2351cbc419590796e90d45b5768/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.147414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552970c6-bd13-47e3-98bc-c57b45dc2a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=acdd3142f6f346e8a1399fe19c8c0ca5, user_id=7adea2351cbc419590796e90d45b5768, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:30.150171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552970c6-bd13-47e3-98bc-c57b45dc2a5d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:30.167251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1210/2373] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:03:30 2022] PUT /identity/v3/projects/acdd3142f6f346e8a1399fe19c8c0ca5/users/7adea2351cbc419590796e90d45b5768/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:30.236470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e740de4-150e-4608-aa87-47de0e0955f3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.236752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e740de4-150e-4608-aa87-47de0e0955f3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.237198 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e740de4-150e-4608-aa87-47de0e0955f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.266005 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-5e740de4-150e-4608-aa87-47de0e0955f3 None None] MFA Rules not processed for user `7adea2351cbc419590796e90d45b5768`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:30.267337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5e740de4-150e-4608-aa87-47de0e0955f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:30.341894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1164/2374] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:30.351591 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552970c6-bd13-47e3-98bc-c57b45dc2a5d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:30.427916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552970c6-bd13-47e3-98bc-c57b45dc2a5d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.432596 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce3631-15ed-41b8-a5af-af42f9b2358b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.433177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce3631-15ed-41b8-a5af-af42f9b2358b admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.433451 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce3631-15ed-41b8-a5af-af42f9b2358b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.433714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce3631-15ed-41b8-a5af-af42f9b2358b admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.434329 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce3631-15ed-41b8-a5af-af42f9b2358b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:30.436866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce3631-15ed-41b8-a5af-af42f9b2358b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:30.470335 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1211/2375] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:03:30 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:30.471258 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e740de4-150e-4608-aa87-47de0e0955f3 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:30.483033 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e740de4-150e-4608-aa87-47de0e0955f3 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.534328 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:30.544813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3254-c543-4386-b47c-e84b1c915ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.545476 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3254-c543-4386-b47c-e84b1c915ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.545683 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3254-c543-4386-b47c-e84b1c915ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.545949 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3254-c543-4386-b47c-e84b1c915ec1 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.546600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f3254-c543-4386-b47c-e84b1c915ec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:30.549641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f3254-c543-4386-b47c-e84b1c915ec1 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:30.573069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1165/2376] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:03:30 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:03:30.576915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce3631-15ed-41b8-a5af-af42f9b2358b admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:30.587375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce3631-15ed-41b8-a5af-af42f9b2358b admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.593285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff09fe-6c5b-4307-acac-9fc2baa5d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.593514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff09fe-6c5b-4307-acac-9fc2baa5d7ba admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.593704 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff09fe-6c5b-4307-acac-9fc2baa5d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.593946 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff09fe-6c5b-4307-acac-9fc2baa5d7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.594593 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff09fe-6c5b-4307-acac-9fc2baa5d7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:30.596232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff09fe-6c5b-4307-acac-9fc2baa5d7ba admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:30.608326 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1212/2377] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:30.612782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3254-c543-4386-b47c-e84b1c915ec1 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:30.623586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3254-c543-4386-b47c-e84b1c915ec1 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.628582 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cbbf0-53f1-4c3b-9b9a-06bd54af082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.629244 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cbbf0-53f1-4c3b-9b9a-06bd54af082c admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.629520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cbbf0-53f1-4c3b-9b9a-06bd54af082c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.629790 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cbbf0-53f1-4c3b-9b9a-06bd54af082c admin admin] PATH_INFO: `/v3/projects/01206b5de6814784a8799b37d03dd608/users/d7c398ea35b64459a10300ef19bb99bc/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.644624 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cbbf0-53f1-4c3b-9b9a-06bd54af082c admin admin] RBAC: Authorizing `identity:create_grant(project_id=01206b5de6814784a8799b37d03dd608, user_id=d7c398ea35b64459a10300ef19bb99bc, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:30.647290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cbbf0-53f1-4c3b-9b9a-06bd54af082c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:30.660561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1166/2378] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:03:30 2022] PUT /identity/v3/projects/01206b5de6814784a8799b37d03dd608/users/d7c398ea35b64459a10300ef19bb99bc/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:03:30.664528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff09fe-6c5b-4307-acac-9fc2baa5d7ba admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:30.675024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff09fe-6c5b-4307-acac-9fc2baa5d7ba admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.725392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:30.732938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc99a0-b603-4e33-98e1-9aabbefd7096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.733538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc99a0-b603-4e33-98e1-9aabbefd7096 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.733813 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc99a0-b603-4e33-98e1-9aabbefd7096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.734104 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc99a0-b603-4e33-98e1-9aabbefd7096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.734734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc99a0-b603-4e33-98e1-9aabbefd7096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:30.736349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc99a0-b603-4e33-98e1-9aabbefd7096 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:30.745872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1213/2379] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:30.750829 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cbbf0-53f1-4c3b-9b9a-06bd54af082c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:30.760896 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cbbf0-53f1-4c3b-9b9a-06bd54af082c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.765644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b56a40-fa72-428e-b4fe-b487bd5aebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.766284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b56a40-fa72-428e-b4fe-b487bd5aebd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.766565 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b56a40-fa72-428e-b4fe-b487bd5aebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.766833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b56a40-fa72-428e-b4fe-b487bd5aebd2 admin admin] PATH_INFO: `/v3/projects/01206b5de6814784a8799b37d03dd608/users/d7c398ea35b64459a10300ef19bb99bc/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.768893 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b56a40-fa72-428e-b4fe-b487bd5aebd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01206b5de6814784a8799b37d03dd608, user_id=d7c398ea35b64459a10300ef19bb99bc, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:30.771492 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b56a40-fa72-428e-b4fe-b487bd5aebd2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:30.782170 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1167/2380] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:03:30 2022] PUT /identity/v3/projects/01206b5de6814784a8799b37d03dd608/users/d7c398ea35b64459a10300ef19bb99bc/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:03:30.791962 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f26749-141a-4dcc-8200-b87b8de5a400 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.792243 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f26749-141a-4dcc-8200-b87b8de5a400 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.792510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f26749-141a-4dcc-8200-b87b8de5a400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:30.821566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-a7f26749-141a-4dcc-8200-b87b8de5a400 None None] MFA Rules not processed for user `d7c398ea35b64459a10300ef19bb99bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:30.822746 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a7f26749-141a-4dcc-8200-b87b8de5a400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:30.890236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1214/2381] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:30.901463 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b56a40-fa72-428e-b4fe-b487bd5aebd2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:30.903296 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-53b56a40-fa72-428e-b4fe-b487bd5aebd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:30.959417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b56a40-fa72-428e-b4fe-b487bd5aebd2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:30.962995 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b083e1-16a3-4349-bdaa-998aa28d7e2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:30.963606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b083e1-16a3-4349-bdaa-998aa28d7e2c service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:30.963889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b083e1-16a3-4349-bdaa-998aa28d7e2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:30.964160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b083e1-16a3-4349-bdaa-998aa28d7e2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:31.020923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b083e1-16a3-4349-bdaa-998aa28d7e2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:31.022265 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b083e1-16a3-4349-bdaa-998aa28d7e2c service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:31.031815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1168/2382] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:32.549948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f26749-141a-4dcc-8200-b87b8de5a400 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:32.551506 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a7f26749-141a-4dcc-8200-b87b8de5a400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:32.564990 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f26749-141a-4dcc-8200-b87b8de5a400 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:32.568860 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929a7d3-1c8c-4053-8f5d-18b0ccfe11ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:32.569609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a7d3-1c8c-4053-8f5d-18b0ccfe11ca service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:32.569996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a7d3-1c8c-4053-8f5d-18b0ccfe11ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:32.570341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a7d3-1c8c-4053-8f5d-18b0ccfe11ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:32.629101 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929a7d3-1c8c-4053-8f5d-18b0ccfe11ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:32.630532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929a7d3-1c8c-4053-8f5d-18b0ccfe11ca service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:32.640269 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1215/2383] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:33.212171 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b083e1-16a3-4349-bdaa-998aa28d7e2c service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:33.221245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b083e1-16a3-4349-bdaa-998aa28d7e2c service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:33.224923 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1e87f-deb8-4354-b16f-a8e55d10d798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:33.225528 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1e87f-deb8-4354-b16f-a8e55d10d798 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:33.225804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1e87f-deb8-4354-b16f-a8e55d10d798 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:33.226073 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1e87f-deb8-4354-b16f-a8e55d10d798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:33.236179 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1e87f-deb8-4354-b16f-a8e55d10d798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:33.237630 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1e87f-deb8-4354-b16f-a8e55d10d798 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:33.248691 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1169/2384] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:37.169732 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929a7d3-1c8c-4053-8f5d-18b0ccfe11ca service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:37.231942 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929a7d3-1c8c-4053-8f5d-18b0ccfe11ca service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:37.237199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:37.237820 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:37.238098 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:37.238369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] PATH_INFO: `/v3/users/41d4a769f828433b9a401a5195a9197b` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:37.239476 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d4a769f828433b9a401a5195a9197b)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:37.241441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:37.261711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:37.287073 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d4a769f828433b9a401a5195a9197b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.287718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d4a769f828433b9a401a5195a9197b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.294486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d4a769f828433b9a401a5195a9197b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.295675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d4a769f828433b9a401a5195a9197b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.307495 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d4a769f828433b9a401a5195a9197b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.319907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1216/2385] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:03:37 2022] DELETE /identity/v3/users/41d4a769f828433b9a401a5195a9197b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:37.324429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692c0b-67dd-4136-ae5f-829ffea8e7b3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:37.324913 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692c0b-67dd-4136-ae5f-829ffea8e7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:37.325010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692c0b-67dd-4136-ae5f-829ffea8e7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:37.344374 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-dd692c0b-67dd-4136-ae5f-829ffea8e7b3 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:37.345575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-dd692c0b-67dd-4136-ae5f-829ffea8e7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:37.386772 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:37.405342 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1170/2386] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:37.415006 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:37.416624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:37.474048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3ebd0-34a7-4187-bc05-2c25b97071ad admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:37.477628 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c56fb-8b96-4403-8ea4-74fa4a840d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:37.478209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c56fb-8b96-4403-8ea4-74fa4a840d8d service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:37.478488 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c56fb-8b96-4403-8ea4-74fa4a840d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:37.478752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c56fb-8b96-4403-8ea4-74fa4a840d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:37.487864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c56fb-8b96-4403-8ea4-74fa4a840d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:37.489382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c56fb-8b96-4403-8ea4-74fa4a840d8d service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:37.499092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1217/2387] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:37.760939 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692c0b-67dd-4136-ae5f-829ffea8e7b3 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:37.762450 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-dd692c0b-67dd-4136-ae5f-829ffea8e7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:37.775485 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692c0b-67dd-4136-ae5f-829ffea8e7b3 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:37.782064 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:37.782713 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:37.783048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:37.783251 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] PATH_INFO: `/v3/projects/60e39841d3994e29b603f8695dbe5b3d` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:37.784600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] RBAC: Authorizing `identity:delete_project(project_id=60e39841d3994e29b603f8695dbe5b3d)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:37.786532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:37.838921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60e39841d3994e29b603f8695dbe5b3d', 'request_id': 'req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.845227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60e39841d3994e29b603f8695dbe5b3d', 'request_id': 'req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.852288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60e39841d3994e29b603f8695dbe5b3d', 'request_id': 'req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.863684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] The token cache is being invalidate because project 60e39841d3994e29b603f8695dbe5b3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:03:37.864011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.864644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.866114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1171/2388] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:03:37 2022] DELETE /identity/v3/projects/60e39841d3994e29b603f8695dbe5b3d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:37.870270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c56fb-8b96-4403-8ea4-74fa4a840d8d service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:37.880376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c56fb-8b96-4403-8ea4-74fa4a840d8d service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:37.936396 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:37.936908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:37.937182 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:37.937447 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] PATH_INFO: `/v3/users/7adea2351cbc419590796e90d45b5768` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:37.938553 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7adea2351cbc419590796e90d45b5768)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:37.940567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:37.982207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:37.987989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7adea2351cbc419590796e90d45b5768'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.988652 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7adea2351cbc419590796e90d45b5768'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.996881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7adea2351cbc419590796e90d45b5768'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:37.997497 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7adea2351cbc419590796e90d45b5768'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.009550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7adea2351cbc419590796e90d45b5768'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.022745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1218/2389] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:03:37 2022] DELETE /identity/v3/users/7adea2351cbc419590796e90d45b5768 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:38.305155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:38.306666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:38.321653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263510a3-3fa4-4224-9f3b-4a6ce30fd34d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:38.379126 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:38.379753 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:38.380050 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:38.380321 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] PATH_INFO: `/v3/projects/acdd3142f6f346e8a1399fe19c8c0ca5` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:38.381412 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acdd3142f6f346e8a1399fe19c8c0ca5)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:38.383289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:38.429874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acdd3142f6f346e8a1399fe19c8c0ca5', 'request_id': 'req-02739651-3c07-4959-9ff4-1e51fa1e3615'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.435411 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acdd3142f6f346e8a1399fe19c8c0ca5', 'request_id': 'req-02739651-3c07-4959-9ff4-1e51fa1e3615'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.443045 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acdd3142f6f346e8a1399fe19c8c0ca5', 'request_id': 'req-02739651-3c07-4959-9ff4-1e51fa1e3615'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.450673 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:38.455243 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] The token cache is being invalidate because project acdd3142f6f346e8a1399fe19c8c0ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:03:38.455700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.456811 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.458477 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1172/2390] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:03:38 2022] DELETE /identity/v3/projects/acdd3142f6f346e8a1399fe19c8c0ca5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:38.462499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:38.463897 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:38.476548 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf507a8-c787-42b0-b4aa-9528cc24c83a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:38.532034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:38.532646 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:38.532919 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:38.533186 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] PATH_INFO: `/v3/users/d7c398ea35b64459a10300ef19bb99bc` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:38.534285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7c398ea35b64459a10300ef19bb99bc)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:38.536340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:38.583385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7c398ea35b64459a10300ef19bb99bc'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.584068 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7c398ea35b64459a10300ef19bb99bc'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.591109 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7c398ea35b64459a10300ef19bb99bc'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.591751 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7c398ea35b64459a10300ef19bb99bc'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.604126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d6b2f350-508d-4be5-b1ee-8fe0f9dc0e7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7c398ea35b64459a10300ef19bb99bc'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:38.617429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1219/2391] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:03:38 2022] DELETE /identity/v3/users/d7c398ea35b64459a10300ef19bb99bc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:38.890489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:38.891986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:38.908597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02739651-3c07-4959-9ff4-1e51fa1e3615 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:38.967689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:38.968266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:38.968543 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:38.968831 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] PATH_INFO: `/v3/projects/01206b5de6814784a8799b37d03dd608` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:38.969937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01206b5de6814784a8799b37d03dd608)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:38.971847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:39.018441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01206b5de6814784a8799b37d03dd608', 'request_id': 'req-a69f963e-ee87-41f8-b879-a105fb86ee46'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:39.023984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01206b5de6814784a8799b37d03dd608', 'request_id': 'req-a69f963e-ee87-41f8-b879-a105fb86ee46'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:39.031798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01206b5de6814784a8799b37d03dd608', 'request_id': 'req-a69f963e-ee87-41f8-b879-a105fb86ee46'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:39.043034 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] The token cache is being invalidate because project 01206b5de6814784a8799b37d03dd608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:03:39.043319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:39.043978 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a69f963e-ee87-41f8-b879-a105fb86ee46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:03:39.046002 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1173/2392] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:03:38 2022] DELETE /identity/v3/projects/01206b5de6814784a8799b37d03dd608 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:39.051235 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56329c42-68e2-4762-b835-9e3de6ebd5af None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:39.051516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56329c42-68e2-4762-b835-9e3de6ebd5af None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:39.051784 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56329c42-68e2-4762-b835-9e3de6ebd5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:39.072144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-56329c42-68e2-4762-b835-9e3de6ebd5af None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:39.073346 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-56329c42-68e2-4762-b835-9e3de6ebd5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:39.104995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:39.136725 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1220/2393] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:39.146003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b692ef-bf3e-4c28-875e-423d45ebf050 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:39.146352 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b692ef-bf3e-4c28-875e-423d45ebf050 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:39.146675 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b692ef-bf3e-4c28-875e-423d45ebf050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:39.169172 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-91b692ef-bf3e-4c28-875e-423d45ebf050 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:39.170872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-91b692ef-bf3e-4c28-875e-423d45ebf050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:39.178721 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1174/2394] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:39.183181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56329c42-68e2-4762-b835-9e3de6ebd5af None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:39.193959 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56329c42-68e2-4762-b835-9e3de6ebd5af None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:39.198442 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4f942-1473-4ad3-bcd6-e4c6c024d845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:39.199025 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4f942-1473-4ad3-bcd6-e4c6c024d845 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:39.199294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4f942-1473-4ad3-bcd6-e4c6c024d845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:39.199557 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4f942-1473-4ad3-bcd6-e4c6c024d845 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:39.200222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4f942-1473-4ad3-bcd6-e4c6c024d845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:03:39.200516 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4f942-1473-4ad3-bcd6-e4c6c024d845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:39.202547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4f942-1473-4ad3-bcd6-e4c6c024d845 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:39.213747 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1221/2395] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:03:39.217932 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b692ef-bf3e-4c28-875e-423d45ebf050 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:39.226941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b692ef-bf3e-4c28-875e-423d45ebf050 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:39.231743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf1b1c-5665-4a46-b062-fa40b91b3f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:39.233001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1b1c-5665-4a46-b062-fa40b91b3f00 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:39.233398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1b1c-5665-4a46-b062-fa40b91b3f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:39.233734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1b1c-5665-4a46-b062-fa40b91b3f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:39.234300 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf1b1c-5665-4a46-b062-fa40b91b3f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:39.236210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf1b1c-5665-4a46-b062-fa40b91b3f00 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:39.262900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1175/2396] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:03:39 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:03:39.266989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4f942-1473-4ad3-bcd6-e4c6c024d845 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:39.276734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4f942-1473-4ad3-bcd6-e4c6c024d845 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:39.333232 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3270a9-b577-4dbb-9227-a32f992de5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:39.333822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3270a9-b577-4dbb-9227-a32f992de5a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:39.334090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3270a9-b577-4dbb-9227-a32f992de5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:39.334355 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3270a9-b577-4dbb-9227-a32f992de5a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:39.334989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3270a9-b577-4dbb-9227-a32f992de5a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:39.336983 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3270a9-b577-4dbb-9227-a32f992de5a0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:39.359434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1222/2397] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:03:39 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:03:39.363546 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf1b1c-5665-4a46-b062-fa40b91b3f00 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:39.373902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf1b1c-5665-4a46-b062-fa40b91b3f00 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:39.378473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff44c5-0965-429a-9fbe-cb59eeac163d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:39.379067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff44c5-0965-429a-9fbe-cb59eeac163d admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:39.379345 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff44c5-0965-429a-9fbe-cb59eeac163d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:39.379869 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff44c5-0965-429a-9fbe-cb59eeac163d admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:39.380281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff44c5-0965-429a-9fbe-cb59eeac163d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:39.381863 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff44c5-0965-429a-9fbe-cb59eeac163d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:39.392277 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1176/2398] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:39.396176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3270a9-b577-4dbb-9227-a32f992de5a0 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:39.406639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3270a9-b577-4dbb-9227-a32f992de5a0 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:39.411163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b130db-f364-44ff-ab4c-d2380a137896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:39.411738 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b130db-f364-44ff-ab4c-d2380a137896 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:39.412049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b130db-f364-44ff-ab4c-d2380a137896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:39.412317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b130db-f364-44ff-ab4c-d2380a137896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:39.412944 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b130db-f364-44ff-ab4c-d2380a137896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:39.414572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b130db-f364-44ff-ab4c-d2380a137896 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:39.425529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1223/2399] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:03:39.429558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff44c5-0965-429a-9fbe-cb59eeac163d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:39.440326 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff44c5-0965-429a-9fbe-cb59eeac163d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:39.444915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cafde-39a3-43e8-9c69-6cd5893af838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:39.445555 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cafde-39a3-43e8-9c69-6cd5893af838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:39.445833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cafde-39a3-43e8-9c69-6cd5893af838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:39.446106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cafde-39a3-43e8-9c69-6cd5893af838 admin admin] PATH_INFO: `/v3/projects/b0b92ab9577946a3af15e4698bd5b1e9/users/20b39293079c4ec68aa1482bc7adee96/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:39.460695 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cafde-39a3-43e8-9c69-6cd5893af838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b92ab9577946a3af15e4698bd5b1e9, user_id=20b39293079c4ec68aa1482bc7adee96, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:39.463336 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cafde-39a3-43e8-9c69-6cd5893af838 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:39.476998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1177/2400] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:03:39 2022] PUT /identity/v3/projects/b0b92ab9577946a3af15e4698bd5b1e9/users/20b39293079c4ec68aa1482bc7adee96/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:03:39.481879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a243e-ddd8-4db2-becc-eb4dad2b4735 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:39.482371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a243e-ddd8-4db2-becc-eb4dad2b4735 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:39.482470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a243e-ddd8-4db2-becc-eb4dad2b4735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:39.501310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-611a243e-ddd8-4db2-becc-eb4dad2b4735 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:39.502494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-611a243e-ddd8-4db2-becc-eb4dad2b4735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:39.561219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1224/2401] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:39.570903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cafde-39a3-43e8-9c69-6cd5893af838 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:39.572605 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6b4cafde-39a3-43e8-9c69-6cd5893af838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:39.605774 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:39.633593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cafde-39a3-43e8-9c69-6cd5893af838 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:39.637216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d2711-2dea-48a6-ae46-2cee2efae66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:39.637814 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d2711-2dea-48a6-ae46-2cee2efae66e service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:39.638084 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d2711-2dea-48a6-ae46-2cee2efae66e service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:39.638348 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d2711-2dea-48a6-ae46-2cee2efae66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:39.647758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d2711-2dea-48a6-ae46-2cee2efae66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:39.649197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d2711-2dea-48a6-ae46-2cee2efae66e service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:39.659453 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1178/2402] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:40.266400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525edf36-6b15-4dc4-b7bb-10268b6c1d33 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:40.266677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525edf36-6b15-4dc4-b7bb-10268b6c1d33 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:40.266940 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525edf36-6b15-4dc4-b7bb-10268b6c1d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:40.295826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-525edf36-6b15-4dc4-b7bb-10268b6c1d33 None None] MFA Rules not processed for user `20b39293079c4ec68aa1482bc7adee96`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:40.297201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-525edf36-6b15-4dc4-b7bb-10268b6c1d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:40.330882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:40.372961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1225/2403] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:40.383762 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d2711-2dea-48a6-ae46-2cee2efae66e service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:03:40.385230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-031d2711-2dea-48a6-ae46-2cee2efae66e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:40.444534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d2711-2dea-48a6-ae46-2cee2efae66e service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:03:40.448219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcc041-f2f5-4351-ae06-cdf82c5a5e6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:03:40.448839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcc041-f2f5-4351-ae06-cdf82c5a5e6b service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:40.449085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcc041-f2f5-4351-ae06-cdf82c5a5e6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:40.449354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcc041-f2f5-4351-ae06-cdf82c5a5e6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:40.459025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcc041-f2f5-4351-ae06-cdf82c5a5e6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:03:40.460545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcc041-f2f5-4351-ae06-cdf82c5a5e6b service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:03:40.470820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1179/2404] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:03:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:03:41.834114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba7051-1400-4e98-a513-fc2b20e76150 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:41.834620 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba7051-1400-4e98-a513-fc2b20e76150 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:41.834728 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba7051-1400-4e98-a513-fc2b20e76150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:41.854002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-03ba7051-1400-4e98-a513-fc2b20e76150 None None] MFA Rules not processed for user `20b39293079c4ec68aa1482bc7adee96`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:41.854908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-03ba7051-1400-4e98-a513-fc2b20e76150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:41.862605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1226/2405] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:03:50.466067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=40 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:51.848337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=15 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:03:54.792129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31fbf5-3508-473f-8f81-5e20ae9ced34 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:03:54.792674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31fbf5-3508-473f-8f81-5e20ae9ced34 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:03:54.792674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31fbf5-3508-473f-8f81-5e20ae9ced34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:03:54.811820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6f31fbf5-3508-473f-8f81-5e20ae9ced34 None None] MFA Rules not processed for user `20b39293079c4ec68aa1482bc7adee96`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:03:54.813107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6f31fbf5-3508-473f-8f81-5e20ae9ced34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:03:54.820012 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1180/2406] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:04:04.806580 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:04:16.140177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eddb17-22c4-4430-8172-d8d74b0b28d1 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:04:16.141265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eddb17-22c4-4430-8172-d8d74b0b28d1 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:04:16.141534 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eddb17-22c4-4430-8172-d8d74b0b28d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:04:16.182275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-94eddb17-22c4-4430-8172-d8d74b0b28d1 None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:04:16.183416 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-94eddb17-22c4-4430-8172-d8d74b0b28d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:04:16.266631 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1227/2407] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Oct 14 18:04:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:04:16.316925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31fbf5-3508-473f-8f81-5e20ae9ced34 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:04:16.318386 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6f31fbf5-3508-473f-8f81-5e20ae9ced34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:04:16.402079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31fbf5-3508-473f-8f81-5e20ae9ced34 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:04:16.408166 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f3022-0312-48f9-b754-e71577c070bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:04:16.409121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f3022-0312-48f9-b754-e71577c070bb service swift] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:04:16.409121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f3022-0312-48f9-b754-e71577c070bb service swift] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:04:16.409525 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f3022-0312-48f9-b754-e71577c070bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:04:16.419016 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f3022-0312-48f9-b754-e71577c070bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:04:16.423988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f3022-0312-48f9-b754-e71577c070bb service swift] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:04:16.434238 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1181/2408] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 18:04:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 18:04:20.916808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ff83-5f47-4cb5-9175-02cd8d2bad3f None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:04:20.917291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ff83-5f47-4cb5-9175-02cd8d2bad3f None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:04:20.917392 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ff83-5f47-4cb5-9175-02cd8d2bad3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:04:20.938548 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2337ff83-5f47-4cb5-9175-02cd8d2bad3f None None] MFA Rules not processed for user `3a0657648d244f12ab76c068b38e641b`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:04:20.940169 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2337ff83-5f47-4cb5-9175-02cd8d2bad3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:04:21.002120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1228/2409] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:04:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:04:26.440572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=32 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:04:30.995927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=57 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:04:32.765067 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df8bef-472c-4890-b0ec-5dfe1c791f8e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:04:32.765536 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df8bef-472c-4890-b0ec-5dfe1c791f8e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:04:32.765820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df8bef-472c-4890-b0ec-5dfe1c791f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:04:32.787212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a8df8bef-472c-4890-b0ec-5dfe1c791f8e None None] MFA Rules not processed for user `d549c67a35a94c65a5cc2a3f3ec83474`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:04:32.788430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a8df8bef-472c-4890-b0ec-5dfe1c791f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:04:32.805308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1182/2410] 10.222.0.27 () {58 vars in 967 bytes} [Fri Oct 14 18:04:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:04:32.811650 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ff83-5f47-4cb5-9175-02cd8d2bad3f None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:04:32.826148 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ff83-5f47-4cb5-9175-02cd8d2bad3f None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:04:32.831692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7756f-99e7-4092-b3cf-256796153c07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec9c3fcf7964640a5eeabc01cd97617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:04:32.832350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7756f-99e7-4092-b3cf-256796153c07 service swift] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:04:32.832810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7756f-99e7-4092-b3cf-256796153c07 service swift] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:04:32.833250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7756f-99e7-4092-b3cf-256796153c07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:04:32.842424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7756f-99e7-4092-b3cf-256796153c07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:04:32.845618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7756f-99e7-4092-b3cf-256796153c07 service swift] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:04:32.855211 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1229/2411] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Oct 14 18:04:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 18:04:34.769021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df8bef-472c-4890-b0ec-5dfe1c791f8e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:04:34.770602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a8df8bef-472c-4890-b0ec-5dfe1c791f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:04:34.854193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7756f-99e7-4092-b3cf-256796153c07 service swift] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:04:34.863443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df8bef-472c-4890-b0ec-5dfe1c791f8e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:04:34.869891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7756f-99e7-4092-b3cf-256796153c07 service swift] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:04:34.871657 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceebcd6-6414-440e-9056-ee06ded7cca9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:04:34.873070 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceebcd6-6414-440e-9056-ee06ded7cca9 service placement] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:04:34.873419 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceebcd6-6414-440e-9056-ee06ded7cca9 service placement] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:04:34.873699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceebcd6-6414-440e-9056-ee06ded7cca9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:04:34.876886 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-fceebcd6-6414-440e-9056-ee06ded7cca9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:04:34.877446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:04:34.878242 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:04:34.878695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:04:34.879122 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] PATH_INFO: `/v3/users/20b39293079c4ec68aa1482bc7adee96` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:04:34.880462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b39293079c4ec68aa1482bc7adee96)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:04:34.883139 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:04:34.899826 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceebcd6-6414-440e-9056-ee06ded7cca9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:04:34.901731 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceebcd6-6414-440e-9056-ee06ded7cca9 service placement] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:04:34.914093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1183/2412] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:04:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:04:34.956195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b39293079c4ec68aa1482bc7adee96'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:04:34.956862 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b39293079c4ec68aa1482bc7adee96'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:04:34.964142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b39293079c4ec68aa1482bc7adee96'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:04:34.965376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b39293079c4ec68aa1482bc7adee96'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:04:34.981197 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b39293079c4ec68aa1482bc7adee96'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:04:34.995024 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1230/2413] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:04:34 2022] DELETE /identity/v3/users/20b39293079c4ec68aa1482bc7adee96 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:04:35.177427 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceebcd6-6414-440e-9056-ee06ded7cca9 service placement] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:04:35.188794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceebcd6-6414-440e-9056-ee06ded7cca9 service placement] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:04:35.266094 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:04:35.266892 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:04:35.267291 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:04:35.267578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] PATH_INFO: `/v3/projects/b0b92ab9577946a3af15e4698bd5b1e9` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:04:35.268984 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b92ab9577946a3af15e4698bd5b1e9)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:04:35.270901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:04:35.323147 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b92ab9577946a3af15e4698bd5b1e9', 'request_id': 'req-8a32405f-955b-4e94-a651-4f0280b217d0'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:04:35.329057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b92ab9577946a3af15e4698bd5b1e9', 'request_id': 'req-8a32405f-955b-4e94-a651-4f0280b217d0'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:04:35.337072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b92ab9577946a3af15e4698bd5b1e9', 'request_id': 'req-8a32405f-955b-4e94-a651-4f0280b217d0'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:04:35.351323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] The token cache is being invalidate because project b0b92ab9577946a3af15e4698bd5b1e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:04:35.351607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:04:35.352319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:04:35.353925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1184/2414] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:04:35 2022] DELETE /identity/v3/projects/b0b92ab9577946a3af15e4698bd5b1e9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:04:44.991508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:04:45.347523 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:13.586896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:13.588328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:13.645226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f877e-519f-44db-b5c7-c5e071dc556d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:13.693545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78835476-2af1-40a2-9367-2a7bc821f3b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:13.694149 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78835476-2af1-40a2-9367-2a7bc821f3b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:13.694423 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78835476-2af1-40a2-9367-2a7bc821f3b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:13.694691 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78835476-2af1-40a2-9367-2a7bc821f3b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:13.696446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-78835476-2af1-40a2-9367-2a7bc821f3b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:13.770186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:13.772193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:13.779091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78835476-2af1-40a2-9367-2a7bc821f3b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:13.780437 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78835476-2af1-40a2-9367-2a7bc821f3b4 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:13.784232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32405f-955b-4e94-a651-4f0280b217d0 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:13.788059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734cb782-864a-41ad-be42-b90416a03d65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:13.788655 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734cb782-864a-41ad-be42-b90416a03d65 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:13.788929 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734cb782-864a-41ad-be42-b90416a03d65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:13.789197 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734cb782-864a-41ad-be42-b90416a03d65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:13.790885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-734cb782-864a-41ad-be42-b90416a03d65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:13.800474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1231/2415] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:13.851711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734cb782-864a-41ad-be42-b90416a03d65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:13.853264 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734cb782-864a-41ad-be42-b90416a03d65 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:13.862558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1185/2416] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:05:21.767976 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64371075-6213-4e6d-95f1-c0710f63d23c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:21.768464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64371075-6213-4e6d-95f1-c0710f63d23c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:21.768583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64371075-6213-4e6d-95f1-c0710f63d23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:21.788317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-64371075-6213-4e6d-95f1-c0710f63d23c None None] MFA Rules not processed for user `072cedf1a6a24f6fa7d7ed92981344dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:05:21.789490 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-64371075-6213-4e6d-95f1-c0710f63d23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:21.796200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1232/2417] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:05:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:05:21.832284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734cb782-864a-41ad-be42-b90416a03d65 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:21.840971 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734cb782-864a-41ad-be42-b90416a03d65 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:21.844577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559246b1-4d8f-49b2-b759-9900d254c1f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:21.845164 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559246b1-4d8f-49b2-b759-9900d254c1f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:21.845440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559246b1-4d8f-49b2-b759-9900d254c1f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:21.845707 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559246b1-4d8f-49b2-b759-9900d254c1f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:21.847414 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-559246b1-4d8f-49b2-b759-9900d254c1f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:21.861287 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559246b1-4d8f-49b2-b759-9900d254c1f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:21.862693 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559246b1-4d8f-49b2-b759-9900d254c1f1 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:21.874155 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1186/2418] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:22.610734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64371075-6213-4e6d-95f1-c0710f63d23c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:22.612179 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-64371075-6213-4e6d-95f1-c0710f63d23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:22.649315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:22.684027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64371075-6213-4e6d-95f1-c0710f63d23c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:22.689227 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429971d7-aeeb-4b00-9467-c4fef603c0c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:22.689824 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429971d7-aeeb-4b00-9467-c4fef603c0c3 service placement] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:22.690100 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429971d7-aeeb-4b00-9467-c4fef603c0c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:22.690365 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429971d7-aeeb-4b00-9467-c4fef603c0c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:22.692079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-429971d7-aeeb-4b00-9467-c4fef603c0c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:22.703701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429971d7-aeeb-4b00-9467-c4fef603c0c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:22.705295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429971d7-aeeb-4b00-9467-c4fef603c0c3 service placement] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:22.715826 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1233/2419] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:29.028286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559246b1-4d8f-49b2-b759-9900d254c1f1 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:29.030028 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-559246b1-4d8f-49b2-b759-9900d254c1f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:29.108285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559246b1-4d8f-49b2-b759-9900d254c1f1 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:29.111803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd7879-cf78-47b4-b3b1-3bdbfcffd0df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:29.112417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd7879-cf78-47b4-b3b1-3bdbfcffd0df service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:29.112689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd7879-cf78-47b4-b3b1-3bdbfcffd0df service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:29.112952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd7879-cf78-47b4-b3b1-3bdbfcffd0df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:29.115943 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-62cd7879-cf78-47b4-b3b1-3bdbfcffd0df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:29.177517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd7879-cf78-47b4-b3b1-3bdbfcffd0df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:29.178965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd7879-cf78-47b4-b3b1-3bdbfcffd0df service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:29.189346 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1187/2420] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:32.711546 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=18 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:33.359753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429971d7-aeeb-4b00-9467-c4fef603c0c3 service placement] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:33.369922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429971d7-aeeb-4b00-9467-c4fef603c0c3 service placement] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:33.373517 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a526bc5-c1d0-4571-a18c-ddda492d73c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:33.374102 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a526bc5-c1d0-4571-a18c-ddda492d73c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:33.374378 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a526bc5-c1d0-4571-a18c-ddda492d73c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:33.374642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a526bc5-c1d0-4571-a18c-ddda492d73c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:33.376363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3a526bc5-c1d0-4571-a18c-ddda492d73c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:33.439263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a526bc5-c1d0-4571-a18c-ddda492d73c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:33.440726 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a526bc5-c1d0-4571-a18c-ddda492d73c4 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:33.450874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1234/2421] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:05:36.592806 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd7879-cf78-47b4-b3b1-3bdbfcffd0df service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:36.594395 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-62cd7879-cf78-47b4-b3b1-3bdbfcffd0df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:36.604722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=83 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:36.612638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd7879-cf78-47b4-b3b1-3bdbfcffd0df service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:36.636642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09fb23-3afb-40dc-af2d-c99dae06d6c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:36.637516 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09fb23-3afb-40dc-af2d-c99dae06d6c8 service placement] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:36.637715 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09fb23-3afb-40dc-af2d-c99dae06d6c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:36.638248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09fb23-3afb-40dc-af2d-c99dae06d6c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:36.642574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ed09fb23-3afb-40dc-af2d-c99dae06d6c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:36.663932 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a526bc5-c1d0-4571-a18c-ddda492d73c4 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:36.665348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3a526bc5-c1d0-4571-a18c-ddda492d73c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:36.667431 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09fb23-3afb-40dc-af2d-c99dae06d6c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:36.669110 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09fb23-3afb-40dc-af2d-c99dae06d6c8 service placement] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:36.687602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a526bc5-c1d0-4571-a18c-ddda492d73c4 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:36.689934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1188/2422] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:36.694607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:36.696874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:36.697250 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:36.697621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] PATH_INFO: `/v3/users/3a0657648d244f12ab76c068b38e641b` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:36.699000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0657648d244f12ab76c068b38e641b)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:36.701706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:36.771003 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0657648d244f12ab76c068b38e641b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:36.772189 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0657648d244f12ab76c068b38e641b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:36.785508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0657648d244f12ab76c068b38e641b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:36.786337 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0657648d244f12ab76c068b38e641b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:36.803244 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-75b92fd8-f616-44be-a23b-40fec0329bd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0657648d244f12ab76c068b38e641b'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:36.820962 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1235/2423] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:05:36 2022] DELETE /identity/v3/users/3a0657648d244f12ab76c068b38e641b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:05:37.036499 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09fb23-3afb-40dc-af2d-c99dae06d6c8 service placement] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:37.057052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09fb23-3afb-40dc-af2d-c99dae06d6c8 service placement] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:37.116470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:37.117316 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.117618 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.117889 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] PATH_INFO: `/v3/projects/bfdb342c57b144a384e9938fa4c6e0bf` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.119533 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfdb342c57b144a384e9938fa4c6e0bf)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:37.121453 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:37.170937 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfdb342c57b144a384e9938fa4c6e0bf', 'request_id': 'req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:37.176706 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfdb342c57b144a384e9938fa4c6e0bf', 'request_id': 'req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:37.187503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfdb342c57b144a384e9938fa4c6e0bf', 'request_id': 'req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:37.201964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] The token cache is being invalidate because project bfdb342c57b144a384e9938fa4c6e0bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:05:37.202247 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:37.202939 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0fcb8c26-ed77-4f88-bee8-5eac09649ee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:37.204479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1189/2424] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:05:37 2022] DELETE /identity/v3/projects/bfdb342c57b144a384e9938fa4c6e0bf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:05:37.209413 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287aac2d-a27d-43a3-8da9-4387e397b4fc None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.209697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287aac2d-a27d-43a3-8da9-4387e397b4fc None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.209966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287aac2d-a27d-43a3-8da9-4387e397b4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.229877 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-287aac2d-a27d-43a3-8da9-4387e397b4fc None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:05:37.231204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-287aac2d-a27d-43a3-8da9-4387e397b4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:37.292494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1236/2425] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:05:37.301459 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a6247-ca96-4d13-9eca-589f6f44289c None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.301739 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a6247-ca96-4d13-9eca-589f6f44289c None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.302036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a6247-ca96-4d13-9eca-589f6f44289c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.322637 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-568a6247-ca96-4d13-9eca-589f6f44289c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:05:37.323835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-568a6247-ca96-4d13-9eca-589f6f44289c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:37.331611 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1190/2426] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:05:37.336528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287aac2d-a27d-43a3-8da9-4387e397b4fc None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:37.346338 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287aac2d-a27d-43a3-8da9-4387e397b4fc None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:37.351702 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa10d0-d866-4bb1-b20f-eac53722f53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:37.352351 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa10d0-d866-4bb1-b20f-eac53722f53a admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.352623 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa10d0-d866-4bb1-b20f-eac53722f53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.352914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa10d0-d866-4bb1-b20f-eac53722f53a admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.353554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa10d0-d866-4bb1-b20f-eac53722f53a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:05:37.353848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa10d0-d866-4bb1-b20f-eac53722f53a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:37.355510 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa10d0-d866-4bb1-b20f-eac53722f53a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:37.357142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:37.370709 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1237/2427] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:05:37.375102 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a6247-ca96-4d13-9eca-589f6f44289c None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:37.384629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a6247-ca96-4d13-9eca-589f6f44289c None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:37.389306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6f268-f9ea-4aaf-8a9e-0254ab1ac29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:37.389902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6f268-f9ea-4aaf-8a9e-0254ab1ac29e admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.390174 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6f268-f9ea-4aaf-8a9e-0254ab1ac29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.390442 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6f268-f9ea-4aaf-8a9e-0254ab1ac29e admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.391061 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6f268-f9ea-4aaf-8a9e-0254ab1ac29e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:37.393017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6f268-f9ea-4aaf-8a9e-0254ab1ac29e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:37.429412 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1191/2428] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:05:37 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:05:37.433823 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa10d0-d866-4bb1-b20f-eac53722f53a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:37.444028 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa10d0-d866-4bb1-b20f-eac53722f53a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:37.500298 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf42a8-76db-46f3-9a98-2b39cf06a53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:37.500892 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf42a8-76db-46f3-9a98-2b39cf06a53d admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.501167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf42a8-76db-46f3-9a98-2b39cf06a53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.501436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf42a8-76db-46f3-9a98-2b39cf06a53d admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.502061 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf42a8-76db-46f3-9a98-2b39cf06a53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:37.504030 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf42a8-76db-46f3-9a98-2b39cf06a53d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:37.526304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1238/2429] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:05:37 2022] POST /identity/v3/users => generated 278 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:05:37.531225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6f268-f9ea-4aaf-8a9e-0254ab1ac29e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:37.541734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6f268-f9ea-4aaf-8a9e-0254ab1ac29e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:37.546297 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd43ca-b59e-4c5a-8b58-02a313e5d02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:37.546868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd43ca-b59e-4c5a-8b58-02a313e5d02c admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.547144 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd43ca-b59e-4c5a-8b58-02a313e5d02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.547417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd43ca-b59e-4c5a-8b58-02a313e5d02c admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.548076 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfd43ca-b59e-4c5a-8b58-02a313e5d02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:37.549666 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfd43ca-b59e-4c5a-8b58-02a313e5d02c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:37.559765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1192/2430] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:05:37.563913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf42a8-76db-46f3-9a98-2b39cf06a53d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:37.574494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf42a8-76db-46f3-9a98-2b39cf06a53d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:37.579102 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bc688-9a2f-4751-8f62-9eb10ff0886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:37.579692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bc688-9a2f-4751-8f62-9eb10ff0886c admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.579995 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bc688-9a2f-4751-8f62-9eb10ff0886c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.580266 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bc688-9a2f-4751-8f62-9eb10ff0886c admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.580896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bc688-9a2f-4751-8f62-9eb10ff0886c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:37.582473 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bc688-9a2f-4751-8f62-9eb10ff0886c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:37.593749 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1239/2431] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:05:37.597947 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd43ca-b59e-4c5a-8b58-02a313e5d02c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:37.614498 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd43ca-b59e-4c5a-8b58-02a313e5d02c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:37.619496 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158fe5-1e2c-4bbb-9a29-f93f50aa81ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:37.620181 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158fe5-1e2c-4bbb-9a29-f93f50aa81ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.620460 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158fe5-1e2c-4bbb-9a29-f93f50aa81ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.620787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158fe5-1e2c-4bbb-9a29-f93f50aa81ea admin admin] PATH_INFO: `/v3/projects/151abf77d291473398e585cc78fc87d6/users/bfb463d0a4e245ff8a8865d384083c66/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.635835 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158fe5-1e2c-4bbb-9a29-f93f50aa81ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=151abf77d291473398e585cc78fc87d6, user_id=bfb463d0a4e245ff8a8865d384083c66, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:37.638538 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158fe5-1e2c-4bbb-9a29-f93f50aa81ea admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:37.651711 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1193/2432] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:05:37 2022] PUT /identity/v3/projects/151abf77d291473398e585cc78fc87d6/users/bfb463d0a4e245ff8a8865d384083c66/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:05:37.662181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde5d39-4f51-44e8-b010-c947147e5780 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.662485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde5d39-4f51-44e8-b010-c947147e5780 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.662754 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde5d39-4f51-44e8-b010-c947147e5780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.691060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7fde5d39-4f51-44e8-b010-c947147e5780 None None] MFA Rules not processed for user `bfb463d0a4e245ff8a8865d384083c66`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:05:37.692291 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7fde5d39-4f51-44e8-b010-c947147e5780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:37.755350 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1240/2433] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:05:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:05:37.760470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158fe5-1e2c-4bbb-9a29-f93f50aa81ea admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:37.769259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:37.825651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158fe5-1e2c-4bbb-9a29-f93f50aa81ea admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:37.830329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc6465-649f-4436-8bdf-d4416e57fb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:37.830933 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc6465-649f-4436-8bdf-d4416e57fb2d admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.831210 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc6465-649f-4436-8bdf-d4416e57fb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.831484 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc6465-649f-4436-8bdf-d4416e57fb2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.832139 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc6465-649f-4436-8bdf-d4416e57fb2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:37.834046 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc6465-649f-4436-8bdf-d4416e57fb2d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:37.860429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1194/2434] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:05:37 2022] POST /identity/v3/projects => generated 357 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:05:37.864699 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde5d39-4f51-44e8-b010-c947147e5780 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:37.875916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde5d39-4f51-44e8-b010-c947147e5780 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:37.934217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672f76a-fea8-431e-983e-cc419e8743b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:37.934828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672f76a-fea8-431e-983e-cc419e8743b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.935096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672f76a-fea8-431e-983e-cc419e8743b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.935359 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672f76a-fea8-431e-983e-cc419e8743b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.936044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672f76a-fea8-431e-983e-cc419e8743b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:37.938205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672f76a-fea8-431e-983e-cc419e8743b4 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:37.948215 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:37.964575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1241/2435] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:05:37 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:05:37.968284 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc6465-649f-4436-8bdf-d4416e57fb2d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:37.979831 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc6465-649f-4436-8bdf-d4416e57fb2d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:37.984501 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44adf0-a17d-4690-9f30-d69f2109a083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:37.985100 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44adf0-a17d-4690-9f30-d69f2109a083 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:37.985376 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44adf0-a17d-4690-9f30-d69f2109a083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:37.985765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44adf0-a17d-4690-9f30-d69f2109a083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:37.986408 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44adf0-a17d-4690-9f30-d69f2109a083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:37.988173 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44adf0-a17d-4690-9f30-d69f2109a083 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:37.999626 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1195/2436] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 18:05:38.003805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672f76a-fea8-431e-983e-cc419e8743b4 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:38.014428 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672f76a-fea8-431e-983e-cc419e8743b4 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:38.019384 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f03d7-eb4c-4f58-b802-493566e89592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:38.020049 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f03d7-eb4c-4f58-b802-493566e89592 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:38.020319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f03d7-eb4c-4f58-b802-493566e89592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:38.020587 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f03d7-eb4c-4f58-b802-493566e89592 admin admin] PATH_INFO: `/v3/projects/34f8e3832c5f4b97b02dac225a11f68d/users/f3a4bab94af647e7b889fba8f1b753c3/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:38.035864 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f03d7-eb4c-4f58-b802-493566e89592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f8e3832c5f4b97b02dac225a11f68d, user_id=f3a4bab94af647e7b889fba8f1b753c3, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:38.038692 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f03d7-eb4c-4f58-b802-493566e89592 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:38.052744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1242/2437] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:05:38 2022] PUT /identity/v3/projects/34f8e3832c5f4b97b02dac225a11f68d/users/f3a4bab94af647e7b889fba8f1b753c3/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:05:38.057458 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44adf0-a17d-4690-9f30-d69f2109a083 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:38.068356 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44adf0-a17d-4690-9f30-d69f2109a083 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:38.124659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61734142-28bb-4798-9db4-1ad62b14b2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:38.125274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61734142-28bb-4798-9db4-1ad62b14b2aa admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:38.125552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61734142-28bb-4798-9db4-1ad62b14b2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:38.125823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61734142-28bb-4798-9db4-1ad62b14b2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:38.126552 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61734142-28bb-4798-9db4-1ad62b14b2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:38.128110 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61734142-28bb-4798-9db4-1ad62b14b2aa admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:38.138167 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1196/2438] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:05:38.142275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f03d7-eb4c-4f58-b802-493566e89592 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:38.153144 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f03d7-eb4c-4f58-b802-493566e89592 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:38.157865 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306fc0e-e707-49a5-8b4d-3d4725195b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:38.158564 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306fc0e-e707-49a5-8b4d-3d4725195b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:38.158846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306fc0e-e707-49a5-8b4d-3d4725195b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:38.159121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306fc0e-e707-49a5-8b4d-3d4725195b9c admin admin] PATH_INFO: `/v3/projects/34f8e3832c5f4b97b02dac225a11f68d/users/f3a4bab94af647e7b889fba8f1b753c3/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:38.161261 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306fc0e-e707-49a5-8b4d-3d4725195b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f8e3832c5f4b97b02dac225a11f68d, user_id=f3a4bab94af647e7b889fba8f1b753c3, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:38.164543 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306fc0e-e707-49a5-8b4d-3d4725195b9c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:38.174799 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1243/2439] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:05:38 2022] PUT /identity/v3/projects/34f8e3832c5f4b97b02dac225a11f68d/users/f3a4bab94af647e7b889fba8f1b753c3/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:05:38.185131 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042bec-c603-4ad2-acb0-f5f23011f780 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:38.185524 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042bec-c603-4ad2-acb0-f5f23011f780 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:38.185825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042bec-c603-4ad2-acb0-f5f23011f780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:38.213613 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-92042bec-c603-4ad2-acb0-f5f23011f780 None None] MFA Rules not processed for user `f3a4bab94af647e7b889fba8f1b753c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:05:38.214808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-92042bec-c603-4ad2-acb0-f5f23011f780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:38.283355 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1197/2440] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:05:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:05:38.295062 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306fc0e-e707-49a5-8b4d-3d4725195b9c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:38.296814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-6306fc0e-e707-49a5-8b4d-3d4725195b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:38.354349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306fc0e-e707-49a5-8b4d-3d4725195b9c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:38.358010 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c951-6d91-4f23-a4fe-29db286e3e39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:38.358639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c951-6d91-4f23-a4fe-29db286e3e39 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:38.358935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c951-6d91-4f23-a4fe-29db286e3e39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:38.359205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c951-6d91-4f23-a4fe-29db286e3e39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:38.369491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c951-6d91-4f23-a4fe-29db286e3e39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:38.370918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c951-6d91-4f23-a4fe-29db286e3e39 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:38.380280 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1244/2441] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:38.458018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042bec-c603-4ad2-acb0-f5f23011f780 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:38.463694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:38.470398 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042bec-c603-4ad2-acb0-f5f23011f780 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:38.474407 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7615d-1e98-4b3e-ad5d-3d90dd68878c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:38.475004 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7615d-1e98-4b3e-ad5d-3d90dd68878c service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:38.475280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7615d-1e98-4b3e-ad5d-3d90dd68878c service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:38.475569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7615d-1e98-4b3e-ad5d-3d90dd68878c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:38.532532 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7615d-1e98-4b3e-ad5d-3d90dd68878c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:38.533958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7615d-1e98-4b3e-ad5d-3d90dd68878c service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:38.542867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1198/2442] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:39.680786 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c951-6d91-4f23-a4fe-29db286e3e39 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:39.682457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b7f0c951-6d91-4f23-a4fe-29db286e3e39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:39.740927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c951-6d91-4f23-a4fe-29db286e3e39 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:39.744763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb2aab-4dbb-4696-a028-beecf06d1437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:39.745363 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb2aab-4dbb-4696-a028-beecf06d1437 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:39.745639 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb2aab-4dbb-4696-a028-beecf06d1437 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:39.745908 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb2aab-4dbb-4696-a028-beecf06d1437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:39.747621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9deb2aab-4dbb-4696-a028-beecf06d1437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:39.821645 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb2aab-4dbb-4696-a028-beecf06d1437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:39.823798 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb2aab-4dbb-4696-a028-beecf06d1437 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:39.839262 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1245/2443] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:48.538791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=27 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:49.829920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:INSERT=4,keystone:SELECT=93 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:49.906466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7615d-1e98-4b3e-ad5d-3d90dd68878c service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:49.916464 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7615d-1e98-4b3e-ad5d-3d90dd68878c service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:49.920053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e0560-c86d-4252-9ec3-5309aacd8d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:49.920643 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e0560-c86d-4252-9ec3-5309aacd8d10 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:49.920919 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e0560-c86d-4252-9ec3-5309aacd8d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:49.921190 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e0560-c86d-4252-9ec3-5309aacd8d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:49.922899 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5c4e0560-c86d-4252-9ec3-5309aacd8d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:49.981030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e0560-c86d-4252-9ec3-5309aacd8d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:49.982452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e0560-c86d-4252-9ec3-5309aacd8d10 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:49.991306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1199/2444] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:50.891637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb2aab-4dbb-4696-a028-beecf06d1437 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:50.901124 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb2aab-4dbb-4696-a028-beecf06d1437 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:50.905858 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc2d02-fc7d-4b63-b9af-a2dbe5ffda56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:50.906572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc2d02-fc7d-4b63-b9af-a2dbe5ffda56 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:50.907195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc2d02-fc7d-4b63-b9af-a2dbe5ffda56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:50.907485 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc2d02-fc7d-4b63-b9af-a2dbe5ffda56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:50.909698 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-9cdc2d02-fc7d-4b63-b9af-a2dbe5ffda56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:50.922041 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc2d02-fc7d-4b63-b9af-a2dbe5ffda56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:50.924669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc2d02-fc7d-4b63-b9af-a2dbe5ffda56 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:50.935580 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1246/2445] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:53.097771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e0560-c86d-4252-9ec3-5309aacd8d10 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:53.099537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-5c4e0560-c86d-4252-9ec3-5309aacd8d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:53.184906 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e0560-c86d-4252-9ec3-5309aacd8d10 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:53.190672 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837fcf3f-dd4b-47d1-b5ab-880ef75769e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:53.191272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fcf3f-dd4b-47d1-b5ab-880ef75769e9 service nova] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:53.191557 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fcf3f-dd4b-47d1-b5ab-880ef75769e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:53.191828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fcf3f-dd4b-47d1-b5ab-880ef75769e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:53.193885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-837fcf3f-dd4b-47d1-b5ab-880ef75769e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:53.208177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837fcf3f-dd4b-47d1-b5ab-880ef75769e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:53.209817 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837fcf3f-dd4b-47d1-b5ab-880ef75769e9 service nova] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:53.222030 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1200/2446] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:53.469000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc2d02-fc7d-4b63-b9af-a2dbe5ffda56 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:53.546991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc2d02-fc7d-4b63-b9af-a2dbe5ffda56 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:53.552296 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:53.552929 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:53.553203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:53.553467 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] PATH_INFO: `/v3/users/bfb463d0a4e245ff8a8865d384083c66` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:53.554605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfb463d0a4e245ff8a8865d384083c66)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:53.557139 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:53.612812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfb463d0a4e245ff8a8865d384083c66'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:53.613464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfb463d0a4e245ff8a8865d384083c66'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:53.621466 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfb463d0a4e245ff8a8865d384083c66'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:53.622044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfb463d0a4e245ff8a8865d384083c66'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:53.636675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfb463d0a4e245ff8a8865d384083c66'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:53.653927 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1247/2447] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:05:53 2022] DELETE /identity/v3/users/bfb463d0a4e245ff8a8865d384083c66 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:05:53.659158 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cb94e-7a8d-4b6b-8fae-9b193031b0ed None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:53.659503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cb94e-7a8d-4b6b-8fae-9b193031b0ed None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:53.659839 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cb94e-7a8d-4b6b-8fae-9b193031b0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:53.682941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1d2cb94e-7a8d-4b6b-8fae-9b193031b0ed None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:05:53.684360 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1d2cb94e-7a8d-4b6b-8fae-9b193031b0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:53.743831 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1201/2448] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:05:53.753536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:53.754915 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:53.811842 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a7619-f70f-49f7-b035-a30fe0024800 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:53.815689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7813342-6b61-4185-a55b-2efe56631920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:53.816318 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7813342-6b61-4185-a55b-2efe56631920 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:53.816594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7813342-6b61-4185-a55b-2efe56631920 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:53.816859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7813342-6b61-4185-a55b-2efe56631920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:53.826585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7813342-6b61-4185-a55b-2efe56631920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:53.828054 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7813342-6b61-4185-a55b-2efe56631920 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:53.837444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1248/2449] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:54.116627 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cb94e-7a8d-4b6b-8fae-9b193031b0ed None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:54.118097 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1d2cb94e-7a8d-4b6b-8fae-9b193031b0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:54.130991 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cb94e-7a8d-4b6b-8fae-9b193031b0ed None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:54.135785 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:54.136469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:54.136748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:54.137071 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] PATH_INFO: `/v3/projects/151abf77d291473398e585cc78fc87d6` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:54.138195 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=151abf77d291473398e585cc78fc87d6)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:54.140261 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:54.162463 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:54.188825 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '151abf77d291473398e585cc78fc87d6', 'request_id': 'req-25efc422-230e-427f-835c-81a3a3aedc1b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.194329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '151abf77d291473398e585cc78fc87d6', 'request_id': 'req-25efc422-230e-427f-835c-81a3a3aedc1b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.202227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '151abf77d291473398e585cc78fc87d6', 'request_id': 'req-25efc422-230e-427f-835c-81a3a3aedc1b'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.214231 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] The token cache is being invalidate because project 151abf77d291473398e585cc78fc87d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:05:54.214516 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.215157 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.216679 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1202/2450] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:05:54 2022] DELETE /identity/v3/projects/151abf77d291473398e585cc78fc87d6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:05:54.220584 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7813342-6b61-4185-a55b-2efe56631920 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:54.230535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7813342-6b61-4185-a55b-2efe56631920 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:54.235221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:54.287997 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:54.288621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:54.288901 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:54.289174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] PATH_INFO: `/v3/users/f3a4bab94af647e7b889fba8f1b753c3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:54.290364 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a4bab94af647e7b889fba8f1b753c3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:54.292583 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:54.338394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a4bab94af647e7b889fba8f1b753c3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.339029 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a4bab94af647e7b889fba8f1b753c3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.345753 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a4bab94af647e7b889fba8f1b753c3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.346405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a4bab94af647e7b889fba8f1b753c3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.358028 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a4bab94af647e7b889fba8f1b753c3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.371076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1249/2451] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:05:54 2022] DELETE /identity/v3/users/f3a4bab94af647e7b889fba8f1b753c3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:05:54.734007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:54.735632 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:54.748803 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efc422-230e-427f-835c-81a3a3aedc1b admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:54.828955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:54.829597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:54.829873 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:54.830142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] PATH_INFO: `/v3/projects/34f8e3832c5f4b97b02dac225a11f68d` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:54.831259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f8e3832c5f4b97b02dac225a11f68d)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:54.833229 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:54.839435 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:54.840881 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:54.885105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f8e3832c5f4b97b02dac225a11f68d', 'request_id': 'req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.890884 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f8e3832c5f4b97b02dac225a11f68d', 'request_id': 'req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.898910 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f8e3832c5f4b97b02dac225a11f68d', 'request_id': 'req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.901546 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c7999-8245-4fb0-bfbb-527e53944468 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:54.910514 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] The token cache is being invalidate because project 34f8e3832c5f4b97b02dac225a11f68d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:05:54.910798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.911420 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-4d62a2d0-ad0f-4c5f-a705-0b88eadf48c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:05:54.912934 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1203/2452] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:05:54 2022] DELETE /identity/v3/projects/34f8e3832c5f4b97b02dac225a11f68d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:05:54.918340 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10137d87-b255-4d5c-8db3-fa7f94a6781f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:54.918621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10137d87-b255-4d5c-8db3-fa7f94a6781f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:54.919350 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10137d87-b255-4d5c-8db3-fa7f94a6781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:54.941159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-10137d87-b255-4d5c-8db3-fa7f94a6781f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:05:54.942376 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-10137d87-b255-4d5c-8db3-fa7f94a6781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:54.944311 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:54.953491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb290-b1a3-4e19-ba40-5d0c952d6bda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:54.954304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb290-b1a3-4e19-ba40-5d0c952d6bda service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:54.954863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb290-b1a3-4e19-ba40-5d0c952d6bda service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:54.955341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb290-b1a3-4e19-ba40-5d0c952d6bda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:54.957344 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-833eb290-b1a3-4e19-ba40-5d0c952d6bda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:54.976908 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:55.006303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1204/2453] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:05:55.014043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c997cf-0bec-4265-b4f5-03f0bc52f61e None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.014323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c997cf-0bec-4265-b4f5-03f0bc52f61e None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.014597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c997cf-0bec-4265-b4f5-03f0bc52f61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.023717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eb290-b1a3-4e19-ba40-5d0c952d6bda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.025204 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eb290-b1a3-4e19-ba40-5d0c952d6bda service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.035921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1250/2454] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:05:55.036085 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-35c997cf-0bec-4265-b4f5-03f0bc52f61e None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:05:55.037260 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-35c997cf-0bec-4265-b4f5-03f0bc52f61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:55.046068 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1205/2455] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:05:55.050422 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb290-b1a3-4e19-ba40-5d0c952d6bda service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.060734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb290-b1a3-4e19-ba40-5d0c952d6bda service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.065556 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da68b0-122f-4b32-9679-b0fff3aed77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.066143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da68b0-122f-4b32-9679-b0fff3aed77b admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.066414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da68b0-122f-4b32-9679-b0fff3aed77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.066679 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da68b0-122f-4b32-9679-b0fff3aed77b admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.067312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da68b0-122f-4b32-9679-b0fff3aed77b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:05:55.067609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da68b0-122f-4b32-9679-b0fff3aed77b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.069310 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da68b0-122f-4b32-9679-b0fff3aed77b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.085251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1251/2456] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:05:55.086411 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c997cf-0bec-4265-b4f5-03f0bc52f61e None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.095737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c997cf-0bec-4265-b4f5-03f0bc52f61e None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.100456 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0e28d-e654-4bb2-b806-c5a533737526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.101052 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0e28d-e654-4bb2-b806-c5a533737526 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.101324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0e28d-e654-4bb2-b806-c5a533737526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.101591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0e28d-e654-4bb2-b806-c5a533737526 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.102212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0e28d-e654-4bb2-b806-c5a533737526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.104142 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0e28d-e654-4bb2-b806-c5a533737526 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.135317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1206/2457] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:05:55 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:05:55.137163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da68b0-122f-4b32-9679-b0fff3aed77b admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.147424 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da68b0-122f-4b32-9679-b0fff3aed77b admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.206848 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c336403-4437-4717-bcda-b0be92c8edb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.207450 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c336403-4437-4717-bcda-b0be92c8edb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.207731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c336403-4437-4717-bcda-b0be92c8edb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.208034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c336403-4437-4717-bcda-b0be92c8edb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.208675 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c336403-4437-4717-bcda-b0be92c8edb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.210605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c336403-4437-4717-bcda-b0be92c8edb8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.237462 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1252/2458] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:05:55 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 18:05:55.238650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0e28d-e654-4bb2-b806-c5a533737526 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.249468 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0e28d-e654-4bb2-b806-c5a533737526 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.254169 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc2985-7775-4a64-b6e4-f91c0d369cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.254781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc2985-7775-4a64-b6e4-f91c0d369cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.255053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc2985-7775-4a64-b6e4-f91c0d369cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.255323 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc2985-7775-4a64-b6e4-f91c0d369cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.255988 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc2985-7775-4a64-b6e4-f91c0d369cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.257596 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc2985-7775-4a64-b6e4-f91c0d369cb0 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.269401 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1207/2459] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:05:55.273763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c336403-4437-4717-bcda-b0be92c8edb8 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.284695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c336403-4437-4717-bcda-b0be92c8edb8 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.289822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923d29b-e98d-4aee-9764-ae070b6a311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.290434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923d29b-e98d-4aee-9764-ae070b6a311b admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.290710 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923d29b-e98d-4aee-9764-ae070b6a311b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.290981 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923d29b-e98d-4aee-9764-ae070b6a311b admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.291635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923d29b-e98d-4aee-9764-ae070b6a311b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.293918 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923d29b-e98d-4aee-9764-ae070b6a311b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.305529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1253/2460] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:05:55.310447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc2985-7775-4a64-b6e4-f91c0d369cb0 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.321298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc2985-7775-4a64-b6e4-f91c0d369cb0 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.327027 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecffc9-953c-4502-ad0c-41a61ebc4a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.327792 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecffc9-953c-4502-ad0c-41a61ebc4a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.328492 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecffc9-953c-4502-ad0c-41a61ebc4a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.328828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecffc9-953c-4502-ad0c-41a61ebc4a8c admin admin] PATH_INFO: `/v3/projects/81074c01072f4f4e938db4ace1c2c455/users/3ca9874f561e46f6a4933aa3490b4f70/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.349622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecffc9-953c-4502-ad0c-41a61ebc4a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=81074c01072f4f4e938db4ace1c2c455, user_id=3ca9874f561e46f6a4933aa3490b4f70, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.352443 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecffc9-953c-4502-ad0c-41a61ebc4a8c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.368056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1208/2461] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:05:55 2022] PUT /identity/v3/projects/81074c01072f4f4e938db4ace1c2c455/users/3ca9874f561e46f6a4933aa3490b4f70/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:05:55.379614 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fa358-7a39-4a31-91b5-cdcc8691da25 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.380271 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fa358-7a39-4a31-91b5-cdcc8691da25 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.380666 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fa358-7a39-4a31-91b5-cdcc8691da25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.420805 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ee1fa358-7a39-4a31-91b5-cdcc8691da25 None None] MFA Rules not processed for user `3ca9874f561e46f6a4933aa3490b4f70`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:05:55.422053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ee1fa358-7a39-4a31-91b5-cdcc8691da25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:55.471912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:55.491912 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1254/2462] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:05:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:05:55.496987 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecffc9-953c-4502-ad0c-41a61ebc4a8c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.560972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecffc9-953c-4502-ad0c-41a61ebc4a8c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.566663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755b9c3-5f2a-47cd-95a0-0ccba0b78d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.567380 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755b9c3-5f2a-47cd-95a0-0ccba0b78d9c admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.567658 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755b9c3-5f2a-47cd-95a0-0ccba0b78d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.567964 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755b9c3-5f2a-47cd-95a0-0ccba0b78d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.568667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755b9c3-5f2a-47cd-95a0-0ccba0b78d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.570617 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755b9c3-5f2a-47cd-95a0-0ccba0b78d9c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.599909 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1209/2463] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:05:55 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:05:55.604228 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fa358-7a39-4a31-91b5-cdcc8691da25 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.616585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fa358-7a39-4a31-91b5-cdcc8691da25 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.690367 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32eb26-2232-44f1-a816-90aeb11704b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.691013 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32eb26-2232-44f1-a816-90aeb11704b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.691304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32eb26-2232-44f1-a816-90aeb11704b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.691635 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32eb26-2232-44f1-a816-90aeb11704b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.692562 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32eb26-2232-44f1-a816-90aeb11704b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.694532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32eb26-2232-44f1-a816-90aeb11704b2 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.719355 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1255/2464] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:05:55 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:05:55.723366 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755b9c3-5f2a-47cd-95a0-0ccba0b78d9c admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.734413 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755b9c3-5f2a-47cd-95a0-0ccba0b78d9c admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.739185 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b856d27-896e-4f7c-abd8-0fafb2799ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.739797 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b856d27-896e-4f7c-abd8-0fafb2799ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.740123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b856d27-896e-4f7c-abd8-0fafb2799ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.740397 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b856d27-896e-4f7c-abd8-0fafb2799ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.741201 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b856d27-896e-4f7c-abd8-0fafb2799ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.742875 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b856d27-896e-4f7c-abd8-0fafb2799ba8 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.756864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1210/2465] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:05:55.758023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32eb26-2232-44f1-a816-90aeb11704b2 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.768718 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32eb26-2232-44f1-a816-90aeb11704b2 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.773545 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f214e-dae9-46a3-95ba-615ed9533d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.774207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f214e-dae9-46a3-95ba-615ed9533d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.774480 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f214e-dae9-46a3-95ba-615ed9533d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.774752 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f214e-dae9-46a3-95ba-615ed9533d0a admin admin] PATH_INFO: `/v3/projects/e5e184cfc0424bb6ab40775c405187c0/users/92b2864c2dd74f26a07edacb82eb8d64/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.793095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f214e-dae9-46a3-95ba-615ed9533d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e184cfc0424bb6ab40775c405187c0, user_id=92b2864c2dd74f26a07edacb82eb8d64, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.796134 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f214e-dae9-46a3-95ba-615ed9533d0a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.812947 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1256/2466] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:05:55 2022] PUT /identity/v3/projects/e5e184cfc0424bb6ab40775c405187c0/users/92b2864c2dd74f26a07edacb82eb8d64/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:05:55.816910 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b856d27-896e-4f7c-abd8-0fafb2799ba8 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.827656 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b856d27-896e-4f7c-abd8-0fafb2799ba8 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.875176 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:05:55.893306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1300c0-b26f-44af-b773-f1e150189480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.893989 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1300c0-b26f-44af-b773-f1e150189480 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.894272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1300c0-b26f-44af-b773-f1e150189480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.894548 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1300c0-b26f-44af-b773-f1e150189480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.895246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1300c0-b26f-44af-b773-f1e150189480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.896961 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1300c0-b26f-44af-b773-f1e150189480 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.908900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1211/2467] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:05:55.913096 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f214e-dae9-46a3-95ba-615ed9533d0a admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:55.923640 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f214e-dae9-46a3-95ba-615ed9533d0a admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:55.928928 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3be842-d0c8-4727-81d7-20cf4481a312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:55.929615 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3be842-d0c8-4727-81d7-20cf4481a312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:55.929893 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3be842-d0c8-4727-81d7-20cf4481a312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:55.930171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3be842-d0c8-4727-81d7-20cf4481a312 admin admin] PATH_INFO: `/v3/projects/e5e184cfc0424bb6ab40775c405187c0/users/92b2864c2dd74f26a07edacb82eb8d64/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:55.933422 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3be842-d0c8-4727-81d7-20cf4481a312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e184cfc0424bb6ab40775c405187c0, user_id=92b2864c2dd74f26a07edacb82eb8d64, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:55.936480 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3be842-d0c8-4727-81d7-20cf4481a312 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:55.947513 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1257/2468] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:05:55 2022] PUT /identity/v3/projects/e5e184cfc0424bb6ab40775c405187c0/users/92b2864c2dd74f26a07edacb82eb8d64/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:05:56.015842 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0dcc3-1788-40c1-836f-0005b8bfd722 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:56.016230 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0dcc3-1788-40c1-836f-0005b8bfd722 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:56.016502 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0dcc3-1788-40c1-836f-0005b8bfd722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:56.050072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-37d0dcc3-1788-40c1-836f-0005b8bfd722 None None] MFA Rules not processed for user `92b2864c2dd74f26a07edacb82eb8d64`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:05:56.052003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-37d0dcc3-1788-40c1-836f-0005b8bfd722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:56.136901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1212/2469] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:05:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:05:56.148203 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3be842-d0c8-4727-81d7-20cf4481a312 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:56.149938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-1c3be842-d0c8-4727-81d7-20cf4481a312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:05:56.217318 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3be842-d0c8-4727-81d7-20cf4481a312 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:56.221157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43952f03-7a59-43e5-915a-93ef10150b23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:56.221763 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43952f03-7a59-43e5-915a-93ef10150b23 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:56.222030 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43952f03-7a59-43e5-915a-93ef10150b23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:56.222288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43952f03-7a59-43e5-915a-93ef10150b23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:56.233340 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43952f03-7a59-43e5-915a-93ef10150b23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:56.234997 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43952f03-7a59-43e5-915a-93ef10150b23 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:56.244743 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1258/2470] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:05:56.345975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0dcc3-1788-40c1-836f-0005b8bfd722 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:05:56.358450 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0dcc3-1788-40c1-836f-0005b8bfd722 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:05:56.362599 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a4042-47cc-4048-8812-4f732cf45e48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:05:56.363426 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a4042-47cc-4048-8812-4f732cf45e48 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:05:56.364180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a4042-47cc-4048-8812-4f732cf45e48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:05:56.364667 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a4042-47cc-4048-8812-4f732cf45e48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:05:56.443705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a4042-47cc-4048-8812-4f732cf45e48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:05:56.445551 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a4042-47cc-4048-8812-4f732cf45e48 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:05:56.459935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1213/2471] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:05:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:01.713307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43952f03-7a59-43e5-915a-93ef10150b23 service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:01.714987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-43952f03-7a59-43e5-915a-93ef10150b23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:01.761155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:01.780802 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43952f03-7a59-43e5-915a-93ef10150b23 service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:01.785290 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:01.785907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:01.786181 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:01.786445 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] PATH_INFO: `/v3/users/74943fd4c96b43e8a32989e7ed74cdf2` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:01.787544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] RBAC: Authorizing `identity:delete_user(user_id=74943fd4c96b43e8a32989e7ed74cdf2)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:01.789483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:01.835133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74943fd4c96b43e8a32989e7ed74cdf2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:01.835689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74943fd4c96b43e8a32989e7ed74cdf2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:01.843585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74943fd4c96b43e8a32989e7ed74cdf2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:01.844245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74943fd4c96b43e8a32989e7ed74cdf2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:01.856722 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74943fd4c96b43e8a32989e7ed74cdf2'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:01.869152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1259/2472] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:01 2022] DELETE /identity/v3/users/74943fd4c96b43e8a32989e7ed74cdf2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:02.049677 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a4042-47cc-4048-8812-4f732cf45e48 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:02.061472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a4042-47cc-4048-8812-4f732cf45e48 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:02.118915 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:02.119504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:02.119784 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:02.120227 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] PATH_INFO: `/v3/projects/16610892c7d34b28a2ee5313a4bea23b` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:02.121339 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16610892c7d34b28a2ee5313a4bea23b)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:02.123219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:02.133361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=100 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:02.171325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16610892c7d34b28a2ee5313a4bea23b', 'request_id': 'req-3c8f493c-66ef-4d6f-9835-0a8389b8af80'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.176749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16610892c7d34b28a2ee5313a4bea23b', 'request_id': 'req-3c8f493c-66ef-4d6f-9835-0a8389b8af80'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.184193 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16610892c7d34b28a2ee5313a4bea23b', 'request_id': 'req-3c8f493c-66ef-4d6f-9835-0a8389b8af80'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.195952 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] The token cache is being invalidate because project 16610892c7d34b28a2ee5313a4bea23b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:02.196241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.196887 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.198982 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1214/2473] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:02 2022] DELETE /identity/v3/projects/16610892c7d34b28a2ee5313a4bea23b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:02.202955 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:02.204395 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:02.218083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bf64c-16a6-4904-a5ea-89c5d8d5722f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:02.272288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:02.272903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:02.273177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:02.273449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] PATH_INFO: `/v3/users/5bad6b84ac1141618a82f995b3422ddf` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:02.274568 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bad6b84ac1141618a82f995b3422ddf)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:02.277110 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:02.321385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bad6b84ac1141618a82f995b3422ddf'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.322027 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bad6b84ac1141618a82f995b3422ddf'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.329566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bad6b84ac1141618a82f995b3422ddf'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.330191 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bad6b84ac1141618a82f995b3422ddf'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.337219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:02.343917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bad6b84ac1141618a82f995b3422ddf'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.359356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1260/2474] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:02 2022] DELETE /identity/v3/users/5bad6b84ac1141618a82f995b3422ddf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:02.540890 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:02.542550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:02.556620 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f493c-66ef-4d6f-9835-0a8389b8af80 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:02.616111 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:02.616744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:02.617018 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:02.617285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] PATH_INFO: `/v3/projects/90bee264213b45d9a287167aa9d7475e` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:02.618550 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90bee264213b45d9a287167aa9d7475e)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:02.620681 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:02.665757 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90bee264213b45d9a287167aa9d7475e', 'request_id': 'req-b937519b-89ba-4fa6-8347-403d2c4f6953'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.672771 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90bee264213b45d9a287167aa9d7475e', 'request_id': 'req-b937519b-89ba-4fa6-8347-403d2c4f6953'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.679639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90bee264213b45d9a287167aa9d7475e', 'request_id': 'req-b937519b-89ba-4fa6-8347-403d2c4f6953'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.691159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] The token cache is being invalidate because project 90bee264213b45d9a287167aa9d7475e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:02.691487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.692146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b937519b-89ba-4fa6-8347-403d2c4f6953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:02.693635 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1215/2475] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:02 2022] DELETE /identity/v3/projects/90bee264213b45d9a287167aa9d7475e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:06.919081 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:06.920509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:06.934527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91039ee6-af18-4b94-ae30-dc253609db2d admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:06.990874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:06.991491 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:06.991767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:06.992066 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] PATH_INFO: `/v3/users/3ca9874f561e46f6a4933aa3490b4f70` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:06.993170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ca9874f561e46f6a4933aa3490b4f70)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:06.995788 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:07.039459 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ca9874f561e46f6a4933aa3490b4f70'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.040127 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ca9874f561e46f6a4933aa3490b4f70'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.048079 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ca9874f561e46f6a4933aa3490b4f70'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.048701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ca9874f561e46f6a4933aa3490b4f70'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.060648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ca9874f561e46f6a4933aa3490b4f70'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.074116 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1261/2476] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:06 2022] DELETE /identity/v3/users/3ca9874f561e46f6a4933aa3490b4f70 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:07.078527 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa3867-7e0a-4ab0-8c18-10cad7b7be54 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:07.079008 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa3867-7e0a-4ab0-8c18-10cad7b7be54 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:07.079118 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa3867-7e0a-4ab0-8c18-10cad7b7be54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:07.098503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-d4aa3867-7e0a-4ab0-8c18-10cad7b7be54 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:07.099723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d4aa3867-7e0a-4ab0-8c18-10cad7b7be54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:07.123705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:07.159462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1216/2477] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:07.170843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:07.172324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:07.230910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7274e49-7a6b-47be-8362-283f3aa353a4 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:07.234566 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73122479-8c1e-4431-8b3d-69bafb841bdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:07.235164 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73122479-8c1e-4431-8b3d-69bafb841bdc service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:07.235439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73122479-8c1e-4431-8b3d-69bafb841bdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:07.235711 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73122479-8c1e-4431-8b3d-69bafb841bdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:07.245228 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73122479-8c1e-4431-8b3d-69bafb841bdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:07.246658 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73122479-8c1e-4431-8b3d-69bafb841bdc service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:07.255866 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1262/2478] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:07.518216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa3867-7e0a-4ab0-8c18-10cad7b7be54 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:07.519746 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d4aa3867-7e0a-4ab0-8c18-10cad7b7be54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:07.533815 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa3867-7e0a-4ab0-8c18-10cad7b7be54 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:07.538583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:07.539329 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:07.539679 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:07.540029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] PATH_INFO: `/v3/projects/81074c01072f4f4e938db4ace1c2c455` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:07.541766 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=81074c01072f4f4e938db4ace1c2c455)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:07.543674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:07.590207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81074c01072f4f4e938db4ace1c2c455', 'request_id': 'req-78d572e4-3c2a-4bc1-878a-65feb439cbaf'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.595947 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81074c01072f4f4e938db4ace1c2c455', 'request_id': 'req-78d572e4-3c2a-4bc1-878a-65feb439cbaf'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.603637 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81074c01072f4f4e938db4ace1c2c455', 'request_id': 'req-78d572e4-3c2a-4bc1-878a-65feb439cbaf'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.614883 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] The token cache is being invalidate because project 81074c01072f4f4e938db4ace1c2c455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:07.615161 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.615780 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.617266 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1217/2479] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:07 2022] DELETE /identity/v3/projects/81074c01072f4f4e938db4ace1c2c455 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:07.622221 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73122479-8c1e-4431-8b3d-69bafb841bdc service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:07.628094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:07.631933 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73122479-8c1e-4431-8b3d-69bafb841bdc service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:07.690137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:07.690745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:07.691042 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:07.691309 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] PATH_INFO: `/v3/users/92b2864c2dd74f26a07edacb82eb8d64` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:07.692434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b2864c2dd74f26a07edacb82eb8d64)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:07.694324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:07.740911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b2864c2dd74f26a07edacb82eb8d64'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.741558 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b2864c2dd74f26a07edacb82eb8d64'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.749265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b2864c2dd74f26a07edacb82eb8d64'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.749922 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b2864c2dd74f26a07edacb82eb8d64'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.761641 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-54ce7112-fc7f-4fbd-94f1-abf1b0acec66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b2864c2dd74f26a07edacb82eb8d64'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:07.775931 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1263/2480] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:07 2022] DELETE /identity/v3/users/92b2864c2dd74f26a07edacb82eb8d64 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:08.051760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.053315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:08.067612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d572e4-3c2a-4bc1-878a-65feb439cbaf admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.124882 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.125541 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.125710 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.125981 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] PATH_INFO: `/v3/projects/e5e184cfc0424bb6ab40775c405187c0` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.127095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e184cfc0424bb6ab40775c405187c0)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.129043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.176651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e184cfc0424bb6ab40775c405187c0', 'request_id': 'req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:08.180462 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:08.183082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e184cfc0424bb6ab40775c405187c0', 'request_id': 'req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:08.192088 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e184cfc0424bb6ab40775c405187c0', 'request_id': 'req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:08.204420 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] The token cache is being invalidate because project e5e184cfc0424bb6ab40775c405187c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:08.204709 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:08.205371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-3252fb55-3cde-4a88-a2a1-40e76dc91ec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:08.206849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1218/2481] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:08 2022] DELETE /identity/v3/projects/e5e184cfc0424bb6ab40775c405187c0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:08.212457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3e5f1-408f-45bb-987d-f3100511e90b None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.212954 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3e5f1-408f-45bb-987d-f3100511e90b None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.213088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3e5f1-408f-45bb-987d-f3100511e90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.233958 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-0cc3e5f1-408f-45bb-987d-f3100511e90b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:08.235145 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-0cc3e5f1-408f-45bb-987d-f3100511e90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:08.293132 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1264/2482] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:08.300765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b08598-a212-4366-b9af-4162883f418f None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.301050 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b08598-a212-4366-b9af-4162883f418f None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.301346 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b08598-a212-4366-b9af-4162883f418f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.321063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-50b08598-a212-4366-b9af-4162883f418f None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:08.323010 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-50b08598-a212-4366-b9af-4162883f418f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:08.331025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1219/2483] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:08.335072 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3e5f1-408f-45bb-987d-f3100511e90b None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.345170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3e5f1-408f-45bb-987d-f3100511e90b None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.349648 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b3259-06fc-495d-8eb3-3caa3b0b0797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.350239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b3259-06fc-495d-8eb3-3caa3b0b0797 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.350508 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b3259-06fc-495d-8eb3-3caa3b0b0797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.350772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b3259-06fc-495d-8eb3-3caa3b0b0797 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.351499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b3259-06fc-495d-8eb3-3caa3b0b0797 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:06:08.351681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b3259-06fc-495d-8eb3-3caa3b0b0797 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.354499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b3259-06fc-495d-8eb3-3caa3b0b0797 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.366035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1265/2484] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:06:08.371138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b08598-a212-4366-b9af-4162883f418f None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.380376 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b08598-a212-4366-b9af-4162883f418f None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.385072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a172cff-2a43-41cf-a738-ae389030042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.385664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a172cff-2a43-41cf-a738-ae389030042e admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.385935 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a172cff-2a43-41cf-a738-ae389030042e admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.386199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a172cff-2a43-41cf-a738-ae389030042e admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.386816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a172cff-2a43-41cf-a738-ae389030042e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.388741 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a172cff-2a43-41cf-a738-ae389030042e admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.415696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1220/2485] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:08 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:08.419863 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b3259-06fc-495d-8eb3-3caa3b0b0797 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.430000 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b3259-06fc-495d-8eb3-3caa3b0b0797 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.438595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:08.487021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b990b44-47f8-462d-9b5e-44fc7da670f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.487609 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b990b44-47f8-462d-9b5e-44fc7da670f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.487917 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b990b44-47f8-462d-9b5e-44fc7da670f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.488208 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b990b44-47f8-462d-9b5e-44fc7da670f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.488829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b990b44-47f8-462d-9b5e-44fc7da670f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.490741 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b990b44-47f8-462d-9b5e-44fc7da670f6 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.512589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1266/2486] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:08 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:08.516821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a172cff-2a43-41cf-a738-ae389030042e admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.527600 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a172cff-2a43-41cf-a738-ae389030042e admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.532278 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24631400-27b9-4637-84d9-9a09b2ac4e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.532867 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24631400-27b9-4637-84d9-9a09b2ac4e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.533151 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24631400-27b9-4637-84d9-9a09b2ac4e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.533424 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24631400-27b9-4637-84d9-9a09b2ac4e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.534066 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24631400-27b9-4637-84d9-9a09b2ac4e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.535644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24631400-27b9-4637-84d9-9a09b2ac4e23 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.546068 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1221/2487] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:08.549925 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b990b44-47f8-462d-9b5e-44fc7da670f6 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.561229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b990b44-47f8-462d-9b5e-44fc7da670f6 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.565795 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f941f-0a71-4b7c-9efa-0a8a1ed8c7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.566373 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f941f-0a71-4b7c-9efa-0a8a1ed8c7bf admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.566644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f941f-0a71-4b7c-9efa-0a8a1ed8c7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.566911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f941f-0a71-4b7c-9efa-0a8a1ed8c7bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.567538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f941f-0a71-4b7c-9efa-0a8a1ed8c7bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.569150 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f941f-0a71-4b7c-9efa-0a8a1ed8c7bf admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.579982 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1267/2488] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:08.584001 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24631400-27b9-4637-84d9-9a09b2ac4e23 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.594680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24631400-27b9-4637-84d9-9a09b2ac4e23 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.599381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0ef06-acc0-40f2-b848-172f51f3ec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.600101 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0ef06-acc0-40f2-b848-172f51f3ec14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.600376 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0ef06-acc0-40f2-b848-172f51f3ec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.600641 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0ef06-acc0-40f2-b848-172f51f3ec14 admin admin] PATH_INFO: `/v3/projects/ab29dc6fd73540b4bf05f2888b203e35/users/ccf93079b1aa44c7bfd3c921898af032/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.615786 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0ef06-acc0-40f2-b848-172f51f3ec14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab29dc6fd73540b4bf05f2888b203e35, user_id=ccf93079b1aa44c7bfd3c921898af032, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.618495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0ef06-acc0-40f2-b848-172f51f3ec14 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.631471 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1222/2489] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:08 2022] PUT /identity/v3/projects/ab29dc6fd73540b4bf05f2888b203e35/users/ccf93079b1aa44c7bfd3c921898af032/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:08.641502 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f87d7b-d646-471c-9d21-252f382b349c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.641777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f87d7b-d646-471c-9d21-252f382b349c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.642074 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f87d7b-d646-471c-9d21-252f382b349c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.670153 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-58f87d7b-d646-471c-9d21-252f382b349c None None] MFA Rules not processed for user `ccf93079b1aa44c7bfd3c921898af032`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:08.671331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-58f87d7b-d646-471c-9d21-252f382b349c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:08.734533 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1268/2490] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:08.738968 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0ef06-acc0-40f2-b848-172f51f3ec14 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.799371 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0ef06-acc0-40f2-b848-172f51f3ec14 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.803950 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bc8b6-34dc-4675-a387-339c90dd5e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.804539 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bc8b6-34dc-4675-a387-339c90dd5e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.804816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bc8b6-34dc-4675-a387-339c90dd5e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.805082 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bc8b6-34dc-4675-a387-339c90dd5e57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.805758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bc8b6-34dc-4675-a387-339c90dd5e57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.807654 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bc8b6-34dc-4675-a387-339c90dd5e57 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.833993 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1223/2491] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:08 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:08.837983 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f87d7b-d646-471c-9d21-252f382b349c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.848404 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f87d7b-d646-471c-9d21-252f382b349c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.904143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd395407-c8f1-41ad-a693-e232da9f90ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.904674 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395407-c8f1-41ad-a693-e232da9f90ac admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.904942 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395407-c8f1-41ad-a693-e232da9f90ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.905212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395407-c8f1-41ad-a693-e232da9f90ac admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.905837 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd395407-c8f1-41ad-a693-e232da9f90ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.907808 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd395407-c8f1-41ad-a693-e232da9f90ac admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.927697 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1269/2492] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:08 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:08.931637 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bc8b6-34dc-4675-a387-339c90dd5e57 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.941941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bc8b6-34dc-4675-a387-339c90dd5e57 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.947136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3850e-4fa9-44ea-9f35-5228a3f32552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.947747 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3850e-4fa9-44ea-9f35-5228a3f32552 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.948129 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3850e-4fa9-44ea-9f35-5228a3f32552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.948418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3850e-4fa9-44ea-9f35-5228a3f32552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.949050 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3850e-4fa9-44ea-9f35-5228a3f32552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.950636 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3850e-4fa9-44ea-9f35-5228a3f32552 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.960972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1224/2493] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:08.964879 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd395407-c8f1-41ad-a693-e232da9f90ac admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:08.974735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd395407-c8f1-41ad-a693-e232da9f90ac admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:08.980349 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1716a-d458-4636-9c80-91ec6af5f288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:08.980931 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1716a-d458-4636-9c80-91ec6af5f288 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:08.981207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1716a-d458-4636-9c80-91ec6af5f288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:08.981470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1716a-d458-4636-9c80-91ec6af5f288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:08.982093 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1716a-d458-4636-9c80-91ec6af5f288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:08.983662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1716a-d458-4636-9c80-91ec6af5f288 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:08.985662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:08.995607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1270/2494] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:08.999621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3850e-4fa9-44ea-9f35-5228a3f32552 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:09.009575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3850e-4fa9-44ea-9f35-5228a3f32552 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:09.014159 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb098f0-418b-426c-b80f-b189cab562cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:09.014795 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb098f0-418b-426c-b80f-b189cab562cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:09.015072 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb098f0-418b-426c-b80f-b189cab562cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:09.015341 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb098f0-418b-426c-b80f-b189cab562cc admin admin] PATH_INFO: `/v3/projects/f28f3dbf6e904d1fba67435049a59e2a/users/160c9f721fb74858aca836db0b4baf41/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:09.030668 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb098f0-418b-426c-b80f-b189cab562cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28f3dbf6e904d1fba67435049a59e2a, user_id=160c9f721fb74858aca836db0b4baf41, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:09.033317 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb098f0-418b-426c-b80f-b189cab562cc admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:09.046588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1225/2495] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:08 2022] PUT /identity/v3/projects/f28f3dbf6e904d1fba67435049a59e2a/users/160c9f721fb74858aca836db0b4baf41/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:09.058844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20922953-e9b6-4163-a3bb-3b7ff45c26fd None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:09.059126 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20922953-e9b6-4163-a3bb-3b7ff45c26fd None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:09.059400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20922953-e9b6-4163-a3bb-3b7ff45c26fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:09.087664 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-20922953-e9b6-4163-a3bb-3b7ff45c26fd None None] MFA Rules not processed for user `160c9f721fb74858aca836db0b4baf41`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:09.088896 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-20922953-e9b6-4163-a3bb-3b7ff45c26fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:09.155499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1271/2496] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:09.159861 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb098f0-418b-426c-b80f-b189cab562cc admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:09.192758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:09.224381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb098f0-418b-426c-b80f-b189cab562cc admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:09.229558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db7965-bfb5-4208-9f5d-958fd8a41425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:09.230152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db7965-bfb5-4208-9f5d-958fd8a41425 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:09.230425 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db7965-bfb5-4208-9f5d-958fd8a41425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:09.230689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db7965-bfb5-4208-9f5d-958fd8a41425 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:09.231302 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db7965-bfb5-4208-9f5d-958fd8a41425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:09.233205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db7965-bfb5-4208-9f5d-958fd8a41425 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:09.259696 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1226/2497] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:09 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:09.263729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20922953-e9b6-4163-a3bb-3b7ff45c26fd None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:09.274844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20922953-e9b6-4163-a3bb-3b7ff45c26fd None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:09.330385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd567b7-4c84-4a0b-af82-68edc2b65c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:09.330984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd567b7-4c84-4a0b-af82-68edc2b65c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:09.331258 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd567b7-4c84-4a0b-af82-68edc2b65c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:09.331527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd567b7-4c84-4a0b-af82-68edc2b65c42 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:09.332201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd567b7-4c84-4a0b-af82-68edc2b65c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:09.334111 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd567b7-4c84-4a0b-af82-68edc2b65c42 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:09.359441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1272/2498] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:09 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:09.363386 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db7965-bfb5-4208-9f5d-958fd8a41425 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:09.373734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db7965-bfb5-4208-9f5d-958fd8a41425 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:09.378239 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696aec0e-f294-49fb-a7b3-40ba6d817473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:09.378819 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696aec0e-f294-49fb-a7b3-40ba6d817473 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:09.379093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696aec0e-f294-49fb-a7b3-40ba6d817473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:09.379359 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696aec0e-f294-49fb-a7b3-40ba6d817473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:09.380015 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696aec0e-f294-49fb-a7b3-40ba6d817473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:09.381579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696aec0e-f294-49fb-a7b3-40ba6d817473 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:09.394998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1227/2499] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:09.396120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd567b7-4c84-4a0b-af82-68edc2b65c42 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:09.406744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd567b7-4c84-4a0b-af82-68edc2b65c42 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:09.411239 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123d775-5cc6-4635-9ef0-58436fa5e1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:09.411898 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123d775-5cc6-4635-9ef0-58436fa5e1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:09.412171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123d775-5cc6-4635-9ef0-58436fa5e1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:09.412436 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123d775-5cc6-4635-9ef0-58436fa5e1cf admin admin] PATH_INFO: `/v3/projects/d75279b4acde45b0a2c4d3f6ca8472ad/users/54709218d3704b9b86168c3e6ac3d1f4/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:09.427394 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123d775-5cc6-4635-9ef0-58436fa5e1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75279b4acde45b0a2c4d3f6ca8472ad, user_id=54709218d3704b9b86168c3e6ac3d1f4, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:09.430035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123d775-5cc6-4635-9ef0-58436fa5e1cf admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:09.442528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1273/2500] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:09 2022] PUT /identity/v3/projects/d75279b4acde45b0a2c4d3f6ca8472ad/users/54709218d3704b9b86168c3e6ac3d1f4/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:06:09.446591 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696aec0e-f294-49fb-a7b3-40ba6d817473 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:09.456260 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696aec0e-f294-49fb-a7b3-40ba6d817473 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:09.512445 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a156736-3652-4d75-949d-2ccdcf521bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:09.513041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a156736-3652-4d75-949d-2ccdcf521bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:09.513313 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a156736-3652-4d75-949d-2ccdcf521bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:09.513577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a156736-3652-4d75-949d-2ccdcf521bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:09.514223 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a156736-3652-4d75-949d-2ccdcf521bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:09.515796 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a156736-3652-4d75-949d-2ccdcf521bd9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:09.526096 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1228/2501] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:09.530183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123d775-5cc6-4635-9ef0-58436fa5e1cf admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:09.541056 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123d775-5cc6-4635-9ef0-58436fa5e1cf admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:09.545578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336fb61-ce86-4ac2-8b26-71cb98e3ae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:09.546207 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336fb61-ce86-4ac2-8b26-71cb98e3ae62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:09.546474 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336fb61-ce86-4ac2-8b26-71cb98e3ae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:09.546738 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336fb61-ce86-4ac2-8b26-71cb98e3ae62 admin admin] PATH_INFO: `/v3/projects/d75279b4acde45b0a2c4d3f6ca8472ad/users/54709218d3704b9b86168c3e6ac3d1f4/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:09.548628 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336fb61-ce86-4ac2-8b26-71cb98e3ae62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75279b4acde45b0a2c4d3f6ca8472ad, user_id=54709218d3704b9b86168c3e6ac3d1f4, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:09.551217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336fb61-ce86-4ac2-8b26-71cb98e3ae62 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:09.561772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1274/2502] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:09 2022] PUT /identity/v3/projects/d75279b4acde45b0a2c4d3f6ca8472ad/users/54709218d3704b9b86168c3e6ac3d1f4/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:09.572015 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf17207-3a5d-4562-b40c-cc0b97d1ee92 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:09.572324 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf17207-3a5d-4562-b40c-cc0b97d1ee92 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:09.572597 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf17207-3a5d-4562-b40c-cc0b97d1ee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:09.600064 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-adf17207-3a5d-4562-b40c-cc0b97d1ee92 None None] MFA Rules not processed for user `54709218d3704b9b86168c3e6ac3d1f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:09.601259 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-adf17207-3a5d-4562-b40c-cc0b97d1ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:09.672737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1229/2503] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:06:09.684773 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336fb61-ce86-4ac2-8b26-71cb98e3ae62 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:09.687053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e336fb61-ce86-4ac2-8b26-71cb98e3ae62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:09.732028 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:09.747441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336fb61-ce86-4ac2-8b26-71cb98e3ae62 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:09.751121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af6fdd-7906-4a38-a9db-12e04ae0a88b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:09.751717 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af6fdd-7906-4a38-a9db-12e04ae0a88b service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:09.752034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af6fdd-7906-4a38-a9db-12e04ae0a88b service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:09.752304 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af6fdd-7906-4a38-a9db-12e04ae0a88b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:09.761730 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af6fdd-7906-4a38-a9db-12e04ae0a88b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:09.763188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af6fdd-7906-4a38-a9db-12e04ae0a88b service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:09.772386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1275/2504] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:10.264574 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf17207-3a5d-4562-b40c-cc0b97d1ee92 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:10.265957 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-adf17207-3a5d-4562-b40c-cc0b97d1ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:10.278134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf17207-3a5d-4562-b40c-cc0b97d1ee92 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:10.281650 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0924c7-bf7c-4373-b77a-6deb2660e0e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:10.282237 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0924c7-bf7c-4373-b77a-6deb2660e0e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:10.282512 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0924c7-bf7c-4373-b77a-6deb2660e0e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:10.282781 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0924c7-bf7c-4373-b77a-6deb2660e0e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:10.303372 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:10.340212 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0924c7-bf7c-4373-b77a-6deb2660e0e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:10.341625 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0924c7-bf7c-4373-b77a-6deb2660e0e7 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:10.350511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1230/2505] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:14.370089 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af6fdd-7906-4a38-a9db-12e04ae0a88b service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:14.378601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af6fdd-7906-4a38-a9db-12e04ae0a88b service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:14.382183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e2e64-6742-46b0-b0e7-53bc87196073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:14.382768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e2e64-6742-46b0-b0e7-53bc87196073 service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:14.383044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e2e64-6742-46b0-b0e7-53bc87196073 service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:14.383856 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e2e64-6742-46b0-b0e7-53bc87196073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:14.439796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e2e64-6742-46b0-b0e7-53bc87196073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:14.441266 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e2e64-6742-46b0-b0e7-53bc87196073 service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:14.450681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1276/2506] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:06:16.508234 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0924c7-bf7c-4373-b77a-6deb2660e0e7 service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:16.571039 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0924c7-bf7c-4373-b77a-6deb2660e0e7 service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:16.575509 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:16.576156 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:16.576432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:16.576698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] PATH_INFO: `/v3/users/ccf93079b1aa44c7bfd3c921898af032` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:16.577801 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf93079b1aa44c7bfd3c921898af032)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:16.579697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:16.626998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf93079b1aa44c7bfd3c921898af032'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:16.627634 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf93079b1aa44c7bfd3c921898af032'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:16.634718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf93079b1aa44c7bfd3c921898af032'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:16.635333 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf93079b1aa44c7bfd3c921898af032'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:16.647041 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf93079b1aa44c7bfd3c921898af032'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:16.659718 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1231/2507] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:16 2022] DELETE /identity/v3/users/ccf93079b1aa44c7bfd3c921898af032 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:16.664920 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b616961-61b1-4fe5-a277-b195ba3d8587 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:16.665414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b616961-61b1-4fe5-a277-b195ba3d8587 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:16.665536 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b616961-61b1-4fe5-a277-b195ba3d8587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:16.685090 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7b616961-61b1-4fe5-a277-b195ba3d8587 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:16.686209 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7b616961-61b1-4fe5-a277-b195ba3d8587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:16.744634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1277/2508] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:16.754615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:16.756078 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:16.802700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:16.816738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5823a69-b01f-49a5-a2de-36012b66ecb9 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:16.820974 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5002d-63d1-4e80-b413-851f8abc9f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:16.821572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5002d-63d1-4e80-b413-851f8abc9f20 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:16.821845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5002d-63d1-4e80-b413-851f8abc9f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:16.822115 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5002d-63d1-4e80-b413-851f8abc9f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:16.831051 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5002d-63d1-4e80-b413-851f8abc9f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:16.832495 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5002d-63d1-4e80-b413-851f8abc9f20 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:16.842062 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1232/2509] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 18:06:17.107655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b616961-61b1-4fe5-a277-b195ba3d8587 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:17.109156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7b616961-61b1-4fe5-a277-b195ba3d8587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:17.121624 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b616961-61b1-4fe5-a277-b195ba3d8587 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:17.126212 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:17.126832 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:17.127102 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:17.127380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] PATH_INFO: `/v3/projects/ab29dc6fd73540b4bf05f2888b203e35` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:17.128503 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab29dc6fd73540b4bf05f2888b203e35)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:17.130423 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:17.177870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab29dc6fd73540b4bf05f2888b203e35', 'request_id': 'req-3cf09f16-eafa-429b-9ebd-f8398895b149'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.183899 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab29dc6fd73540b4bf05f2888b203e35', 'request_id': 'req-3cf09f16-eafa-429b-9ebd-f8398895b149'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.191308 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab29dc6fd73540b4bf05f2888b203e35', 'request_id': 'req-3cf09f16-eafa-429b-9ebd-f8398895b149'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.202936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] The token cache is being invalidate because project ab29dc6fd73540b4bf05f2888b203e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:17.203225 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.203859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.205345 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1278/2510] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:17 2022] DELETE /identity/v3/projects/ab29dc6fd73540b4bf05f2888b203e35 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:17.209603 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5002d-63d1-4e80-b413-851f8abc9f20 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:17.219975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5002d-63d1-4e80-b413-851f8abc9f20 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:17.274601 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:17.275215 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:17.275486 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:17.275750 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] PATH_INFO: `/v3/users/160c9f721fb74858aca836db0b4baf41` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:17.276874 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=160c9f721fb74858aca836db0b4baf41)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:17.278765 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:17.325120 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '160c9f721fb74858aca836db0b4baf41'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.325749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '160c9f721fb74858aca836db0b4baf41'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.332520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '160c9f721fb74858aca836db0b4baf41'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.333146 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '160c9f721fb74858aca836db0b4baf41'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.344808 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '160c9f721fb74858aca836db0b4baf41'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.358907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1233/2511] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:17 2022] DELETE /identity/v3/users/160c9f721fb74858aca836db0b4baf41 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:17.625201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:17.626722 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:17.636148 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:17.640843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09f16-eafa-429b-9ebd-f8398895b149 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:17.700429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:17.701091 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:17.701303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:17.701567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] PATH_INFO: `/v3/projects/f28f3dbf6e904d1fba67435049a59e2a` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:17.702663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f28f3dbf6e904d1fba67435049a59e2a)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:17.704589 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:17.752571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f28f3dbf6e904d1fba67435049a59e2a', 'request_id': 'req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.757847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f28f3dbf6e904d1fba67435049a59e2a', 'request_id': 'req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.765030 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f28f3dbf6e904d1fba67435049a59e2a', 'request_id': 'req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.777085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] The token cache is being invalidate because project f28f3dbf6e904d1fba67435049a59e2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:17.777369 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.778001 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.779744 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1279/2512] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:17 2022] DELETE /identity/v3/projects/f28f3dbf6e904d1fba67435049a59e2a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:17.783859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:17.785903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:17.800280 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a047a-386c-479d-98fc-01aec939ef3a admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:17.857113 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:17.857737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:17.858015 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:17.858285 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] PATH_INFO: `/v3/users/54709218d3704b9b86168c3e6ac3d1f4` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:17.859384 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54709218d3704b9b86168c3e6ac3d1f4)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:17.861337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:17.865776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:17.908079 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54709218d3704b9b86168c3e6ac3d1f4'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.908736 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54709218d3704b9b86168c3e6ac3d1f4'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.916689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54709218d3704b9b86168c3e6ac3d1f4'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.917361 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54709218d3704b9b86168c3e6ac3d1f4'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.930920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0d1edbbc-b74b-4891-840a-e0489db8a716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54709218d3704b9b86168c3e6ac3d1f4'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:17.943479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1234/2513] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:17 2022] DELETE /identity/v3/users/54709218d3704b9b86168c3e6ac3d1f4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:18.214721 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:18.216314 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:18.229729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9508ae2-dfe1-4d05-b9c0-ba41294aa08f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:18.286020 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:18.286645 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.286923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.287195 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] PATH_INFO: `/v3/projects/d75279b4acde45b0a2c4d3f6ca8472ad` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.288328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75279b4acde45b0a2c4d3f6ca8472ad)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:18.290263 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:18.329143 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:18.338765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75279b4acde45b0a2c4d3f6ca8472ad', 'request_id': 'req-1074b5cc-afe7-441c-bb81-92442e18061a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:18.344278 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75279b4acde45b0a2c4d3f6ca8472ad', 'request_id': 'req-1074b5cc-afe7-441c-bb81-92442e18061a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:18.352528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75279b4acde45b0a2c4d3f6ca8472ad', 'request_id': 'req-1074b5cc-afe7-441c-bb81-92442e18061a'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:18.363240 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] The token cache is being invalidate because project d75279b4acde45b0a2c4d3f6ca8472ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:18.363525 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:18.364163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1074b5cc-afe7-441c-bb81-92442e18061a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:18.365630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1280/2514] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:18 2022] DELETE /identity/v3/projects/d75279b4acde45b0a2c4d3f6ca8472ad => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:06:18.372053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be5558-e092-4c81-9213-c5249d8093f8 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.372337 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be5558-e092-4c81-9213-c5249d8093f8 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.372607 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be5558-e092-4c81-9213-c5249d8093f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.392586 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-c5be5558-e092-4c81-9213-c5249d8093f8 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:18.393787 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-c5be5558-e092-4c81-9213-c5249d8093f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:18.452516 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1235/2515] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:18.462095 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bf1af-58b1-48f4-9964-e95f89a49552 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.462374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bf1af-58b1-48f4-9964-e95f89a49552 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.462642 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bf1af-58b1-48f4-9964-e95f89a49552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.482257 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e35bf1af-58b1-48f4-9964-e95f89a49552 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:18.483444 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e35bf1af-58b1-48f4-9964-e95f89a49552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:18.491254 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1281/2516] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:18.496054 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be5558-e092-4c81-9213-c5249d8093f8 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:18.505788 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be5558-e092-4c81-9213-c5249d8093f8 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:18.511219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78861cd-1f22-47a2-bef9-14b3d289767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:18.511816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78861cd-1f22-47a2-bef9-14b3d289767d admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.512124 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78861cd-1f22-47a2-bef9-14b3d289767d admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.512423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78861cd-1f22-47a2-bef9-14b3d289767d admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.513057 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78861cd-1f22-47a2-bef9-14b3d289767d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:06:18.513354 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78861cd-1f22-47a2-bef9-14b3d289767d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:18.514960 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78861cd-1f22-47a2-bef9-14b3d289767d admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:18.526507 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1236/2517] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:06:18.530777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bf1af-58b1-48f4-9964-e95f89a49552 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:18.539957 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bf1af-58b1-48f4-9964-e95f89a49552 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:18.544468 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f6203-6d08-454c-9ee9-6224c36d09c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:18.545044 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6203-6d08-454c-9ee9-6224c36d09c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.545313 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6203-6d08-454c-9ee9-6224c36d09c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.545576 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6203-6d08-454c-9ee9-6224c36d09c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.546193 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f6203-6d08-454c-9ee9-6224c36d09c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:18.548093 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f6203-6d08-454c-9ee9-6224c36d09c7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:18.576769 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1282/2518] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:18 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:18.581000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78861cd-1f22-47a2-bef9-14b3d289767d admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:18.590722 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78861cd-1f22-47a2-bef9-14b3d289767d admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:18.646405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd490e8-1a68-4d44-8217-8f6c5db0d6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:18.647000 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd490e8-1a68-4d44-8217-8f6c5db0d6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.647272 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd490e8-1a68-4d44-8217-8f6c5db0d6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.647540 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd490e8-1a68-4d44-8217-8f6c5db0d6a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.648385 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd490e8-1a68-4d44-8217-8f6c5db0d6a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:18.650277 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd490e8-1a68-4d44-8217-8f6c5db0d6a2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:18.667275 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:18.673639 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1237/2519] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:18 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:18.677520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f6203-6d08-454c-9ee9-6224c36d09c7 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:18.688532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f6203-6d08-454c-9ee9-6224c36d09c7 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:18.693052 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2084f7-1ffa-4256-947e-21d34b8c2aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:18.693634 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2084f7-1ffa-4256-947e-21d34b8c2aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.693903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2084f7-1ffa-4256-947e-21d34b8c2aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.694167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2084f7-1ffa-4256-947e-21d34b8c2aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.694821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2084f7-1ffa-4256-947e-21d34b8c2aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:18.696618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2084f7-1ffa-4256-947e-21d34b8c2aa9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:18.706319 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1283/2520] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:18.710816 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd490e8-1a68-4d44-8217-8f6c5db0d6a2 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:18.720752 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd490e8-1a68-4d44-8217-8f6c5db0d6a2 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:18.725256 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f21dd8-7512-4f88-b2de-b68cc98859ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:18.725837 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21dd8-7512-4f88-b2de-b68cc98859ec admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.726905 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21dd8-7512-4f88-b2de-b68cc98859ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.727028 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21dd8-7512-4f88-b2de-b68cc98859ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.727638 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f21dd8-7512-4f88-b2de-b68cc98859ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:18.729245 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f21dd8-7512-4f88-b2de-b68cc98859ec admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:18.739480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1238/2521] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:18.744375 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2084f7-1ffa-4256-947e-21d34b8c2aa9 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:18.753988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2084f7-1ffa-4256-947e-21d34b8c2aa9 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:18.758575 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28998f0-b1e4-4ea8-989a-9b71e039dff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:18.759201 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28998f0-b1e4-4ea8-989a-9b71e039dff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.759470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28998f0-b1e4-4ea8-989a-9b71e039dff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.759735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28998f0-b1e4-4ea8-989a-9b71e039dff2 admin admin] PATH_INFO: `/v3/projects/6e3d7591a28d4574bafb2337572f3325/users/aef0bc69628a4ea9a2090fcd35293b17/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.774183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28998f0-b1e4-4ea8-989a-9b71e039dff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3d7591a28d4574bafb2337572f3325, user_id=aef0bc69628a4ea9a2090fcd35293b17, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:18.776914 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28998f0-b1e4-4ea8-989a-9b71e039dff2 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:18.790123 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1284/2522] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:18 2022] PUT /identity/v3/projects/6e3d7591a28d4574bafb2337572f3325/users/aef0bc69628a4ea9a2090fcd35293b17/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:06:18.801363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8400-0f6e-4e94-92ae-dfa1704ef0f3 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.801852 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8400-0f6e-4e94-92ae-dfa1704ef0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.801954 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8400-0f6e-4e94-92ae-dfa1704ef0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.829836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-266d8400-0f6e-4e94-92ae-dfa1704ef0f3 None None] MFA Rules not processed for user `aef0bc69628a4ea9a2090fcd35293b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:18.831089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-266d8400-0f6e-4e94-92ae-dfa1704ef0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:18.892938 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1239/2523] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:18.897332 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28998f0-b1e4-4ea8-989a-9b71e039dff2 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:18.959371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28998f0-b1e4-4ea8-989a-9b71e039dff2 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:18.964076 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd77f6-bee5-4a70-8491-bbea3a672e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:18.964696 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd77f6-bee5-4a70-8491-bbea3a672e3e admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:18.964975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd77f6-bee5-4a70-8491-bbea3a672e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:18.965241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd77f6-bee5-4a70-8491-bbea3a672e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:18.965882 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd77f6-bee5-4a70-8491-bbea3a672e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:18.967834 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd77f6-bee5-4a70-8491-bbea3a672e3e admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:18.994785 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1285/2524] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:18 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:18.998856 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d8400-0f6e-4e94-92ae-dfa1704ef0f3 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:19.009665 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d8400-0f6e-4e94-92ae-dfa1704ef0f3 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:19.067180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01872b8-15ce-4e28-ae72-2e10f3a8de53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:19.067812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01872b8-15ce-4e28-ae72-2e10f3a8de53 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:19.068113 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01872b8-15ce-4e28-ae72-2e10f3a8de53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:19.068382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01872b8-15ce-4e28-ae72-2e10f3a8de53 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:19.069907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01872b8-15ce-4e28-ae72-2e10f3a8de53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:19.072025 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01872b8-15ce-4e28-ae72-2e10f3a8de53 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:19.094141 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1240/2525] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:18 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:19.098125 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd77f6-bee5-4a70-8491-bbea3a672e3e admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:19.109230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd77f6-bee5-4a70-8491-bbea3a672e3e admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:19.113768 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e0435-8901-4684-8394-0bdc0678e9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:19.114359 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e0435-8901-4684-8394-0bdc0678e9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:19.114637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e0435-8901-4684-8394-0bdc0678e9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:19.114903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e0435-8901-4684-8394-0bdc0678e9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:19.115535 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e0435-8901-4684-8394-0bdc0678e9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:19.117297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e0435-8901-4684-8394-0bdc0678e9f4 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:19.128222 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1286/2526] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 18:06:19.132579 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01872b8-15ce-4e28-ae72-2e10f3a8de53 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:19.142894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01872b8-15ce-4e28-ae72-2e10f3a8de53 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:19.147470 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74ca56-83f4-409d-ba2a-57dee8ed7029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:19.148642 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74ca56-83f4-409d-ba2a-57dee8ed7029 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:19.148916 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74ca56-83f4-409d-ba2a-57dee8ed7029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:19.149191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74ca56-83f4-409d-ba2a-57dee8ed7029 admin admin] PATH_INFO: `/v3/projects/108b81d03cdd41798f875c679b4f5967/users/eca1dfa099cb448cbd8e2bd373289d93/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:19.164832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74ca56-83f4-409d-ba2a-57dee8ed7029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108b81d03cdd41798f875c679b4f5967, user_id=eca1dfa099cb448cbd8e2bd373289d93, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:19.167644 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74ca56-83f4-409d-ba2a-57dee8ed7029 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:19.179999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1241/2527] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:19 2022] PUT /identity/v3/projects/108b81d03cdd41798f875c679b4f5967/users/eca1dfa099cb448cbd8e2bd373289d93/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:19.184315 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e0435-8901-4684-8394-0bdc0678e9f4 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:19.194429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e0435-8901-4684-8394-0bdc0678e9f4 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:19.227681 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:19.252847 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fcb4b7-38b6-409d-8940-47a1e970bf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:19.253440 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcb4b7-38b6-409d-8940-47a1e970bf40 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:19.253716 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcb4b7-38b6-409d-8940-47a1e970bf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:19.253985 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcb4b7-38b6-409d-8940-47a1e970bf40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:19.254637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fcb4b7-38b6-409d-8940-47a1e970bf40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:19.256265 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fcb4b7-38b6-409d-8940-47a1e970bf40 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:19.265828 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1287/2528] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:19.270106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74ca56-83f4-409d-ba2a-57dee8ed7029 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:19.280829 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74ca56-83f4-409d-ba2a-57dee8ed7029 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:19.285379 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eeb31-efe6-4f5a-b90a-9d95d887b3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:19.286021 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eeb31-efe6-4f5a-b90a-9d95d887b3d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:19.286306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eeb31-efe6-4f5a-b90a-9d95d887b3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:19.286578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eeb31-efe6-4f5a-b90a-9d95d887b3d7 admin admin] PATH_INFO: `/v3/projects/108b81d03cdd41798f875c679b4f5967/users/eca1dfa099cb448cbd8e2bd373289d93/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:19.288606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1eeb31-efe6-4f5a-b90a-9d95d887b3d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108b81d03cdd41798f875c679b4f5967, user_id=eca1dfa099cb448cbd8e2bd373289d93, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:19.291232 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1eeb31-efe6-4f5a-b90a-9d95d887b3d7 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:19.301301 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1242/2529] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:19 2022] PUT /identity/v3/projects/108b81d03cdd41798f875c679b4f5967/users/eca1dfa099cb448cbd8e2bd373289d93/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:19.313414 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ca881-47ae-43b1-91de-5d2d276ed719 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:19.313693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ca881-47ae-43b1-91de-5d2d276ed719 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:19.313967 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ca881-47ae-43b1-91de-5d2d276ed719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:19.342660 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-ec7ca881-47ae-43b1-91de-5d2d276ed719 None None] MFA Rules not processed for user `eca1dfa099cb448cbd8e2bd373289d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:19.343800 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-ec7ca881-47ae-43b1-91de-5d2d276ed719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:19.411133 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1288/2530] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:06:19.422670 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eeb31-efe6-4f5a-b90a-9d95d887b3d7 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:19.424409 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-ba1eeb31-efe6-4f5a-b90a-9d95d887b3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:19.458751 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:19.485378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eeb31-efe6-4f5a-b90a-9d95d887b3d7 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:19.488966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641d874-a808-43c6-9292-11bf6450f544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:19.489556 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641d874-a808-43c6-9292-11bf6450f544 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:19.489824 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641d874-a808-43c6-9292-11bf6450f544 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:19.490089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641d874-a808-43c6-9292-11bf6450f544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:19.499473 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641d874-a808-43c6-9292-11bf6450f544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:19.500914 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641d874-a808-43c6-9292-11bf6450f544 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:19.510036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1243/2531] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:19.643783 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ca881-47ae-43b1-91de-5d2d276ed719 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:19.708551 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ca881-47ae-43b1-91de-5d2d276ed719 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:19.714013 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:19.714633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:19.714902 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:19.715166 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] PATH_INFO: `/v3/users/aef0bc69628a4ea9a2090fcd35293b17` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:19.716289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef0bc69628a4ea9a2090fcd35293b17)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:19.718184 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:19.764523 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef0bc69628a4ea9a2090fcd35293b17'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:19.765163 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef0bc69628a4ea9a2090fcd35293b17'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:19.771916 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef0bc69628a4ea9a2090fcd35293b17'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:19.773034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef0bc69628a4ea9a2090fcd35293b17'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:19.775400 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:19.787007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef0bc69628a4ea9a2090fcd35293b17'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:19.800499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1289/2532] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:19 2022] DELETE /identity/v3/users/aef0bc69628a4ea9a2090fcd35293b17 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:19.805255 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe4de3-3096-489d-bedd-18d54fc11835 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:19.805748 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe4de3-3096-489d-bedd-18d54fc11835 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:19.805850 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe4de3-3096-489d-bedd-18d54fc11835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:19.825253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-6ffe4de3-3096-489d-bedd-18d54fc11835 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:19.826584 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6ffe4de3-3096-489d-bedd-18d54fc11835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:19.884833 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1244/2533] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:19.894438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:19.895825 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:19.952680 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e260d73-9e78-4680-9316-a1926a083237 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:19.956362 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85b4af-023f-44d3-837c-1733f7c2182f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:19.956968 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85b4af-023f-44d3-837c-1733f7c2182f service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:19.957243 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85b4af-023f-44d3-837c-1733f7c2182f service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:19.957509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85b4af-023f-44d3-837c-1733f7c2182f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:19.967119 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85b4af-023f-44d3-837c-1733f7c2182f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:19.968603 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85b4af-023f-44d3-837c-1733f7c2182f service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:19.978461 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1290/2534] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:20.236973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe4de3-3096-489d-bedd-18d54fc11835 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:20.238472 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-6ffe4de3-3096-489d-bedd-18d54fc11835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:20.253772 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe4de3-3096-489d-bedd-18d54fc11835 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:20.259359 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:20.260153 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:20.260315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:20.260589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] PATH_INFO: `/v3/projects/6e3d7591a28d4574bafb2337572f3325` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:20.261845 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3d7591a28d4574bafb2337572f3325)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:20.263744 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:20.311156 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3d7591a28d4574bafb2337572f3325', 'request_id': 'req-dcfb5268-ac23-44e3-ab74-68a15f47f870'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.317049 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3d7591a28d4574bafb2337572f3325', 'request_id': 'req-dcfb5268-ac23-44e3-ab74-68a15f47f870'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.324672 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3d7591a28d4574bafb2337572f3325', 'request_id': 'req-dcfb5268-ac23-44e3-ab74-68a15f47f870'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.335953 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] The token cache is being invalidate because project 6e3d7591a28d4574bafb2337572f3325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:20.336241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.336893 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.338373 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1245/2535] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:20 2022] DELETE /identity/v3/projects/6e3d7591a28d4574bafb2337572f3325 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:20.342448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85b4af-023f-44d3-837c-1733f7c2182f service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:20.352521 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85b4af-023f-44d3-837c-1733f7c2182f service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:20.408612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:20.409241 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:20.409514 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:20.409779 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] PATH_INFO: `/v3/users/eca1dfa099cb448cbd8e2bd373289d93` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:20.410894 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca1dfa099cb448cbd8e2bd373289d93)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:20.412869 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:20.458443 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca1dfa099cb448cbd8e2bd373289d93'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.459092 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca1dfa099cb448cbd8e2bd373289d93'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.466448 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca1dfa099cb448cbd8e2bd373289d93'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.467071 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca1dfa099cb448cbd8e2bd373289d93'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.478765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-1344b81d-f474-45e6-ad4c-6b64bd04c776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca1dfa099cb448cbd8e2bd373289d93'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.486987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:20.493478 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1291/2536] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:20 2022] DELETE /identity/v3/users/eca1dfa099cb448cbd8e2bd373289d93 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:20.773106 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:20.774575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:20.789469 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb5268-ac23-44e3-ab74-68a15f47f870 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:20.841921 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:20.848491 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:20.849160 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:20.849648 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:20.850093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] PATH_INFO: `/v3/projects/108b81d03cdd41798f875c679b4f5967` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:20.852572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=108b81d03cdd41798f875c679b4f5967)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:20.854920 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:20.900375 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108b81d03cdd41798f875c679b4f5967', 'request_id': 'req-c695c785-01dd-4cbd-a94f-e26fdb540d71'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.905684 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108b81d03cdd41798f875c679b4f5967', 'request_id': 'req-c695c785-01dd-4cbd-a94f-e26fdb540d71'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.913593 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108b81d03cdd41798f875c679b4f5967', 'request_id': 'req-c695c785-01dd-4cbd-a94f-e26fdb540d71'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.924480 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] The token cache is being invalidate because project 108b81d03cdd41798f875c679b4f5967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:20.924760 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.925370 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-c695c785-01dd-4cbd-a94f-e26fdb540d71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:20.926818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1246/2537] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:20 2022] DELETE /identity/v3/projects/108b81d03cdd41798f875c679b4f5967 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:20.932190 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607c54c-2161-4cd7-85e4-5eb76b2b119c None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:20.932694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607c54c-2161-4cd7-85e4-5eb76b2b119c None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:20.932814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607c54c-2161-4cd7-85e4-5eb76b2b119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:20.952748 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-7607c54c-2161-4cd7-85e4-5eb76b2b119c None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:20.953857 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-7607c54c-2161-4cd7-85e4-5eb76b2b119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:21.011153 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1292/2538] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:21.018674 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bcd6e-9d31-4487-a2fb-0c0c9a8627c2 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.018955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bcd6e-9d31-4487-a2fb-0c0c9a8627c2 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.019253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bcd6e-9d31-4487-a2fb-0c0c9a8627c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.039225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-326bcd6e-9d31-4487-a2fb-0c0c9a8627c2 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:21.040452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-326bcd6e-9d31-4487-a2fb-0c0c9a8627c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:21.048612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1247/2539] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:21.052663 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607c54c-2161-4cd7-85e4-5eb76b2b119c None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.062236 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607c54c-2161-4cd7-85e4-5eb76b2b119c None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.066693 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681f3bb-52ae-4ff1-9a40-d2ed0f76cff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.067277 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681f3bb-52ae-4ff1-9a40-d2ed0f76cff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.067549 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681f3bb-52ae-4ff1-9a40-d2ed0f76cff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.067833 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681f3bb-52ae-4ff1-9a40-d2ed0f76cff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.068499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681f3bb-52ae-4ff1-9a40-d2ed0f76cff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:06:21.068790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681f3bb-52ae-4ff1-9a40-d2ed0f76cff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.070987 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681f3bb-52ae-4ff1-9a40-d2ed0f76cff0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.082194 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1293/2540] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:06:21.086998 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bcd6e-9d31-4487-a2fb-0c0c9a8627c2 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.096188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bcd6e-9d31-4487-a2fb-0c0c9a8627c2 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.100897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda04d6-cb08-4289-9b4d-a5d215b88115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.101578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda04d6-cb08-4289-9b4d-a5d215b88115 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.101910 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda04d6-cb08-4289-9b4d-a5d215b88115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.102186 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda04d6-cb08-4289-9b4d-a5d215b88115 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.102813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda04d6-cb08-4289-9b4d-a5d215b88115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.104778 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda04d6-cb08-4289-9b4d-a5d215b88115 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.131180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1248/2541] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:21 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:21.135941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681f3bb-52ae-4ff1-9a40-d2ed0f76cff0 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.145764 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681f3bb-52ae-4ff1-9a40-d2ed0f76cff0 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.203205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093245-15ae-4df2-9cee-93b9707ac07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.203731 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093245-15ae-4df2-9cee-93b9707ac07f admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.204060 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093245-15ae-4df2-9cee-93b9707ac07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.204328 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093245-15ae-4df2-9cee-93b9707ac07f admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.204975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95093245-15ae-4df2-9cee-93b9707ac07f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.206907 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95093245-15ae-4df2-9cee-93b9707ac07f admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.227618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1294/2542] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:21 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:21.231832 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda04d6-cb08-4289-9b4d-a5d215b88115 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.242735 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda04d6-cb08-4289-9b4d-a5d215b88115 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.247264 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f4e7e-0109-4541-8ce3-606489e341f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.247846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f4e7e-0109-4541-8ce3-606489e341f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.248152 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f4e7e-0109-4541-8ce3-606489e341f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.248417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f4e7e-0109-4541-8ce3-606489e341f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.249044 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f4e7e-0109-4541-8ce3-606489e341f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.250615 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f4e7e-0109-4541-8ce3-606489e341f8 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.260851 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1249/2543] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:21.264694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093245-15ae-4df2-9cee-93b9707ac07f admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.274323 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093245-15ae-4df2-9cee-93b9707ac07f admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.278872 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433223be-d954-4ec7-a2f0-3d7b7111aca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.279453 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433223be-d954-4ec7-a2f0-3d7b7111aca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.279726 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433223be-d954-4ec7-a2f0-3d7b7111aca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.280032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433223be-d954-4ec7-a2f0-3d7b7111aca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.280662 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433223be-d954-4ec7-a2f0-3d7b7111aca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.282224 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433223be-d954-4ec7-a2f0-3d7b7111aca8 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.292552 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1295/2544] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:21.296487 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f4e7e-0109-4541-8ce3-606489e341f8 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.307191 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f4e7e-0109-4541-8ce3-606489e341f8 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.312069 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0aba9-a1b4-4a93-add4-0d591448f147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.312705 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0aba9-a1b4-4a93-add4-0d591448f147 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.312976 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0aba9-a1b4-4a93-add4-0d591448f147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.313241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0aba9-a1b4-4a93-add4-0d591448f147 admin admin] PATH_INFO: `/v3/projects/b4c20355d95e4ba9a7fa2ecb07cbeeaa/users/d28e22052385434ea86bed76a06e941c/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.329130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0aba9-a1b4-4a93-add4-0d591448f147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c20355d95e4ba9a7fa2ecb07cbeeaa, user_id=d28e22052385434ea86bed76a06e941c, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.331738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0aba9-a1b4-4a93-add4-0d591448f147 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.345090 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1250/2545] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:21 2022] PUT /identity/v3/projects/b4c20355d95e4ba9a7fa2ecb07cbeeaa/users/d28e22052385434ea86bed76a06e941c/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:21.356314 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f4d86-4b76-403e-841f-d29e5e194483 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.356598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f4d86-4b76-403e-841f-d29e5e194483 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.356867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f4d86-4b76-403e-841f-d29e5e194483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.384129 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-404f4d86-4b76-403e-841f-d29e5e194483 None None] MFA Rules not processed for user `d28e22052385434ea86bed76a06e941c`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:21.385331 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-404f4d86-4b76-403e-841f-d29e5e194483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:21.440438 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:21.452021 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1296/2546] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:21.456545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0aba9-a1b4-4a93-add4-0d591448f147 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.518959 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0aba9-a1b4-4a93-add4-0d591448f147 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.524105 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959876ce-2bc9-46bc-bd33-889e28db9271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.524698 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959876ce-2bc9-46bc-bd33-889e28db9271 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.524973 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959876ce-2bc9-46bc-bd33-889e28db9271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.525241 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959876ce-2bc9-46bc-bd33-889e28db9271 admin admin] PATH_INFO: `/v3/projects` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.525860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959876ce-2bc9-46bc-bd33-889e28db9271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.527749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959876ce-2bc9-46bc-bd33-889e28db9271 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.554225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1251/2547] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:21 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:21.558494 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f4d86-4b76-403e-841f-d29e5e194483 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.569030 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f4d86-4b76-403e-841f-d29e5e194483 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.624683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3ba35-7f17-4261-bfb5-ed3e5921b440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.625281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3ba35-7f17-4261-bfb5-ed3e5921b440 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.625554 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3ba35-7f17-4261-bfb5-ed3e5921b440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.625821 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3ba35-7f17-4261-bfb5-ed3e5921b440 admin admin] PATH_INFO: `/v3/users` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.626433 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3ba35-7f17-4261-bfb5-ed3e5921b440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.628382 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3ba35-7f17-4261-bfb5-ed3e5921b440 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.649871 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1297/2548] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:21 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:21.653776 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959876ce-2bc9-46bc-bd33-889e28db9271 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.663872 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959876ce-2bc9-46bc-bd33-889e28db9271 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.668454 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32baf4a2-9f8a-4d0b-a5ca-026013db2c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.669038 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32baf4a2-9f8a-4d0b-a5ca-026013db2c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.669308 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32baf4a2-9f8a-4d0b-a5ca-026013db2c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.669573 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32baf4a2-9f8a-4d0b-a5ca-026013db2c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.670203 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32baf4a2-9f8a-4d0b-a5ca-026013db2c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.671819 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32baf4a2-9f8a-4d0b-a5ca-026013db2c39 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.673860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:21.683749 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1252/2549] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:21.687798 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3ba35-7f17-4261-bfb5-ed3e5921b440 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.698607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3ba35-7f17-4261-bfb5-ed3e5921b440 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.703200 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4807bb-e51f-49be-91ca-abaa45680ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.703834 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4807bb-e51f-49be-91ca-abaa45680ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.704137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4807bb-e51f-49be-91ca-abaa45680ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.704407 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4807bb-e51f-49be-91ca-abaa45680ea9 admin admin] PATH_INFO: `/v3/projects/9acb1a195d0f4d5486f1ee26dcbd1979/users/c2f30d2b5a274b5388ac3a5a2a5d9d55/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.719729 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4807bb-e51f-49be-91ca-abaa45680ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9acb1a195d0f4d5486f1ee26dcbd1979, user_id=c2f30d2b5a274b5388ac3a5a2a5d9d55, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.722572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4807bb-e51f-49be-91ca-abaa45680ea9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.738449 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1298/2550] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:21 2022] PUT /identity/v3/projects/9acb1a195d0f4d5486f1ee26dcbd1979/users/c2f30d2b5a274b5388ac3a5a2a5d9d55/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 18:06:21.742380 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32baf4a2-9f8a-4d0b-a5ca-026013db2c39 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.752365 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32baf4a2-9f8a-4d0b-a5ca-026013db2c39 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.808761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b399cd-c135-4eca-8d0d-e5b620b5e2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.809365 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b399cd-c135-4eca-8d0d-e5b620b5e2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.809645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b399cd-c135-4eca-8d0d-e5b620b5e2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.809918 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b399cd-c135-4eca-8d0d-e5b620b5e2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.810581 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b399cd-c135-4eca-8d0d-e5b620b5e2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.812187 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b399cd-c135-4eca-8d0d-e5b620b5e2b2 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.822562 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1253/2551] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:21.826901 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4807bb-e51f-49be-91ca-abaa45680ea9 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.837594 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4807bb-e51f-49be-91ca-abaa45680ea9 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:21.842215 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcdd22-ca2f-4dd2-8747-403c2088c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:21.842853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcdd22-ca2f-4dd2-8747-403c2088c9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.843130 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcdd22-ca2f-4dd2-8747-403c2088c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.843405 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcdd22-ca2f-4dd2-8747-403c2088c9d7 admin admin] PATH_INFO: `/v3/projects/9acb1a195d0f4d5486f1ee26dcbd1979/users/c2f30d2b5a274b5388ac3a5a2a5d9d55/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.845377 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcdd22-ca2f-4dd2-8747-403c2088c9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9acb1a195d0f4d5486f1ee26dcbd1979, user_id=c2f30d2b5a274b5388ac3a5a2a5d9d55, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:21.848035 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcdd22-ca2f-4dd2-8747-403c2088c9d7 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:21.858202 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1299/2552] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:21 2022] PUT /identity/v3/projects/9acb1a195d0f4d5486f1ee26dcbd1979/users/c2f30d2b5a274b5388ac3a5a2a5d9d55/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:21.869441 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feaa42a-e670-41c1-a63f-52458de9ad50 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:21.869951 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feaa42a-e670-41c1-a63f-52458de9ad50 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:21.870056 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feaa42a-e670-41c1-a63f-52458de9ad50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:21.898965 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-1feaa42a-e670-41c1-a63f-52458de9ad50 None None] MFA Rules not processed for user `c2f30d2b5a274b5388ac3a5a2a5d9d55`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:21.900206 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-1feaa42a-e670-41c1-a63f-52458de9ad50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:21.967677 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1254/2553] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 18:06:21.979362 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcdd22-ca2f-4dd2-8747-403c2088c9d7 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:21.981108 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c4bcdd22-ca2f-4dd2-8747-403c2088c9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:22.038289 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcdd22-ca2f-4dd2-8747-403c2088c9d7 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:22.041953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d26b5-9a3f-41b6-bec6-bd94c8e4dc3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:22.042542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d26b5-9a3f-41b6-bec6-bd94c8e4dc3a service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:22.042810 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d26b5-9a3f-41b6-bec6-bd94c8e4dc3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:22.043102 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d26b5-9a3f-41b6-bec6-bd94c8e4dc3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:22.098047 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d26b5-9a3f-41b6-bec6-bd94c8e4dc3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:22.099476 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d26b5-9a3f-41b6-bec6-bd94c8e4dc3a service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:22.104013 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:22.108867 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1300/2554] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:24.637972 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feaa42a-e670-41c1-a63f-52458de9ad50 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:24.699868 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feaa42a-e670-41c1-a63f-52458de9ad50 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:24.704286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:24.704907 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:24.705182 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:24.705446 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] PATH_INFO: `/v3/users/d28e22052385434ea86bed76a06e941c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:24.706538 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28e22052385434ea86bed76a06e941c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:24.708451 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:24.730448 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:24.755226 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28e22052385434ea86bed76a06e941c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:24.755888 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28e22052385434ea86bed76a06e941c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:24.762955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28e22052385434ea86bed76a06e941c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:24.763588 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28e22052385434ea86bed76a06e941c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:24.775334 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28e22052385434ea86bed76a06e941c'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:24.787823 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1255/2555] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:24 2022] DELETE /identity/v3/users/d28e22052385434ea86bed76a06e941c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:24.793002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25689c-5bf4-4fb9-9040-f195d01e0665 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:24.793493 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25689c-5bf4-4fb9-9040-f195d01e0665 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:24.793616 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25689c-5bf4-4fb9-9040-f195d01e0665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:24.812542 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-2b25689c-5bf4-4fb9-9040-f195d01e0665 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:24.813734 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2b25689c-5bf4-4fb9-9040-f195d01e0665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:24.871330 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1301/2556] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:24.880865 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:24.882902 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:24.939123 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6824f-6cff-4423-98c4-1667303239c0 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:24.942680 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ffe46-c880-4eba-99db-389e244e4932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:24.943276 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ffe46-c880-4eba-99db-389e244e4932 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:24.943577 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ffe46-c880-4eba-99db-389e244e4932 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:24.943849 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ffe46-c880-4eba-99db-389e244e4932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:24.953387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ffe46-c880-4eba-99db-389e244e4932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:24.954809 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ffe46-c880-4eba-99db-389e244e4932 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:24.964813 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1256/2557] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:25.230023 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25689c-5bf4-4fb9-9040-f195d01e0665 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:25.231527 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-2b25689c-5bf4-4fb9-9040-f195d01e0665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:25.244157 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25689c-5bf4-4fb9-9040-f195d01e0665 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:25.248689 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:25.249895 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:25.250379 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:25.250655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] PATH_INFO: `/v3/projects/b4c20355d95e4ba9a7fa2ecb07cbeeaa` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:25.251792 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c20355d95e4ba9a7fa2ecb07cbeeaa)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:25.254330 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:25.302183 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c20355d95e4ba9a7fa2ecb07cbeeaa', 'request_id': 'req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.308793 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c20355d95e4ba9a7fa2ecb07cbeeaa', 'request_id': 'req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.316248 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c20355d95e4ba9a7fa2ecb07cbeeaa', 'request_id': 'req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.328509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] The token cache is being invalidate because project b4c20355d95e4ba9a7fa2ecb07cbeeaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:25.328796 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.329439 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.330921 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1302/2558] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:25 2022] DELETE /identity/v3/projects/b4c20355d95e4ba9a7fa2ecb07cbeeaa => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:25.334944 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ffe46-c880-4eba-99db-389e244e4932 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:25.344794 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ffe46-c880-4eba-99db-389e244e4932 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:25.400697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:25.401325 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:25.401606 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:25.401879 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] PATH_INFO: `/v3/users/c2f30d2b5a274b5388ac3a5a2a5d9d55` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:25.402992 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f30d2b5a274b5388ac3a5a2a5d9d55)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:25.404986 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:25.448547 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:25.451976 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f30d2b5a274b5388ac3a5a2a5d9d55'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.452975 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f30d2b5a274b5388ac3a5a2a5d9d55'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.461286 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f30d2b5a274b5388ac3a5a2a5d9d55'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.461913 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f30d2b5a274b5388ac3a5a2a5d9d55'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.473734 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-af21838c-8acd-454c-bb6d-e659b4627ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f30d2b5a274b5388ac3a5a2a5d9d55'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.486578 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1257/2559] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:25 2022] DELETE /identity/v3/users/c2f30d2b5a274b5388ac3a5a2a5d9d55 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:25.755152 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:25.756633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:25.772249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1b57b-e58c-4c1a-9ebb-fe180d360a4c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:25.835622 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:25.836317 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:25.836578 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:25.836843 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] PATH_INFO: `/v3/projects/9acb1a195d0f4d5486f1ee26dcbd1979` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:25.837941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9acb1a195d0f4d5486f1ee26dcbd1979)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:25.839831 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:25.869170 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:25.885281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9acb1a195d0f4d5486f1ee26dcbd1979', 'request_id': 'req-c7920b2f-2723-4988-851d-9b0bfdf51be9'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.890612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9acb1a195d0f4d5486f1ee26dcbd1979', 'request_id': 'req-c7920b2f-2723-4988-851d-9b0bfdf51be9'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.897429 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9acb1a195d0f4d5486f1ee26dcbd1979', 'request_id': 'req-c7920b2f-2723-4988-851d-9b0bfdf51be9'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.908706 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] The token cache is being invalidate because project 9acb1a195d0f4d5486f1ee26dcbd1979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:25.908986 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.909598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-c7920b2f-2723-4988-851d-9b0bfdf51be9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:25.911034 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1303/2560] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:25 2022] DELETE /identity/v3/projects/9acb1a195d0f4d5486f1ee26dcbd1979 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:25.916729 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b6566-bc59-49fc-8e7e-8308bfdde5e7 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:25.917305 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b6566-bc59-49fc-8e7e-8308bfdde5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:25.917428 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b6566-bc59-49fc-8e7e-8308bfdde5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:25.937401 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-4e8b6566-bc59-49fc-8e7e-8308bfdde5e7 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:25.938529 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-4e8b6566-bc59-49fc-8e7e-8308bfdde5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:25.996479 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1258/2561] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:26.004325 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28215976-6053-4b8a-b806-a3f43cd52072 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:26.004604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28215976-6053-4b8a-b806-a3f43cd52072 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:26.004870 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28215976-6053-4b8a-b806-a3f43cd52072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:26.025598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-28215976-6053-4b8a-b806-a3f43cd52072 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:26.026777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-28215976-6053-4b8a-b806-a3f43cd52072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:26.034644 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1304/2562] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:26.038966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b6566-bc59-49fc-8e7e-8308bfdde5e7 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:26.048622 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b6566-bc59-49fc-8e7e-8308bfdde5e7 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:26.053149 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c27ac-97bf-4c19-ba19-d32ce719c8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:26.053743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c27ac-97bf-4c19-ba19-d32ce719c8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:26.054876 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c27ac-97bf-4c19-ba19-d32ce719c8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:26.055008 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c27ac-97bf-4c19-ba19-d32ce719c8fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:26.055602 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c27ac-97bf-4c19-ba19-d32ce719c8fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:06:26.055925 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c27ac-97bf-4c19-ba19-d32ce719c8fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:26.057773 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c27ac-97bf-4c19-ba19-d32ce719c8fe admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:26.068970 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1259/2563] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 18:06:26.073938 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28215976-6053-4b8a-b806-a3f43cd52072 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:26.083275 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28215976-6053-4b8a-b806-a3f43cd52072 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:26.088005 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1eaa94-09c4-412e-9e42-4b9e83397968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:26.088618 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eaa94-09c4-412e-9e42-4b9e83397968 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:26.088891 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eaa94-09c4-412e-9e42-4b9e83397968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:26.089156 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eaa94-09c4-412e-9e42-4b9e83397968 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:26.089772 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1eaa94-09c4-412e-9e42-4b9e83397968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:26.091668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1eaa94-09c4-412e-9e42-4b9e83397968 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:26.119936 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1305/2564] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:26 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:26.123999 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c27ac-97bf-4c19-ba19-d32ce719c8fe admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:26.134358 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c27ac-97bf-4c19-ba19-d32ce719c8fe admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:26.190689 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0fd62-dc9e-4c7e-afb3-b58800a0f4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:26.191204 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0fd62-dc9e-4c7e-afb3-b58800a0f4a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:26.191475 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0fd62-dc9e-4c7e-afb3-b58800a0f4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:26.191738 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0fd62-dc9e-4c7e-afb3-b58800a0f4a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:26.192378 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0fd62-dc9e-4c7e-afb3-b58800a0f4a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:26.194273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0fd62-dc9e-4c7e-afb3-b58800a0f4a6 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:26.215870 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1260/2565] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:26 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:26.220074 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1eaa94-09c4-412e-9e42-4b9e83397968 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:26.231121 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1eaa94-09c4-412e-9e42-4b9e83397968 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:26.235595 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f8ec4-2ab3-44ec-97db-497db825d209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:26.236205 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f8ec4-2ab3-44ec-97db-497db825d209 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:26.236492 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f8ec4-2ab3-44ec-97db-497db825d209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:26.236740 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f8ec4-2ab3-44ec-97db-497db825d209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:26.237371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f8ec4-2ab3-44ec-97db-497db825d209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:26.238941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f8ec4-2ab3-44ec-97db-497db825d209 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:26.249567 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1306/2566] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:26.253382 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0fd62-dc9e-4c7e-afb3-b58800a0f4a6 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:26.263610 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0fd62-dc9e-4c7e-afb3-b58800a0f4a6 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:26.268200 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0946452-47a4-4e48-8a30-5d9332035e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:26.268779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0946452-47a4-4e48-8a30-5d9332035e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:26.269053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0946452-47a4-4e48-8a30-5d9332035e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:26.269319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0946452-47a4-4e48-8a30-5d9332035e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:26.269946 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0946452-47a4-4e48-8a30-5d9332035e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:26.271511 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0946452-47a4-4e48-8a30-5d9332035e22 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:26.282687 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1261/2567] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:26.286581 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f8ec4-2ab3-44ec-97db-497db825d209 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:26.297538 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f8ec4-2ab3-44ec-97db-497db825d209 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:26.301998 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f17802-10e3-4ade-ad21-4a71dbfd1e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:26.302619 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f17802-10e3-4ade-ad21-4a71dbfd1e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:26.302890 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f17802-10e3-4ade-ad21-4a71dbfd1e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:26.303155 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f17802-10e3-4ade-ad21-4a71dbfd1e2b admin admin] PATH_INFO: `/v3/projects/4413fbf30c154e7680c5c724eef354b1/users/9da8a2a4b5c34857a75abbb2c255060f/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:26.318470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f17802-10e3-4ade-ad21-4a71dbfd1e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4413fbf30c154e7680c5c724eef354b1, user_id=9da8a2a4b5c34857a75abbb2c255060f, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:26.321661 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f17802-10e3-4ade-ad21-4a71dbfd1e2b admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:26.334281 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1307/2568] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:26 2022] PUT /identity/v3/projects/4413fbf30c154e7680c5c724eef354b1/users/9da8a2a4b5c34857a75abbb2c255060f/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:26.345136 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d601d72-f808-4e55-98b1-06f74b37dec2 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:26.345420 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d601d72-f808-4e55-98b1-06f74b37dec2 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:26.345688 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d601d72-f808-4e55-98b1-06f74b37dec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:26.374432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-9d601d72-f808-4e55-98b1-06f74b37dec2 None None] MFA Rules not processed for user `9da8a2a4b5c34857a75abbb2c255060f`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:26.375631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-9d601d72-f808-4e55-98b1-06f74b37dec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:26.388095 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:26.443289 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1262/2569] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:26.454292 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f17802-10e3-4ade-ad21-4a71dbfd1e2b admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:26.455991 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-c3f17802-10e3-4ade-ad21-4a71dbfd1e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:26.512960 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f17802-10e3-4ade-ad21-4a71dbfd1e2b admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:26.516573 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e6cb7-f002-47c7-9aff-d2ff5c98752b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:26.517161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e6cb7-f002-47c7-9aff-d2ff5c98752b service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:26.517431 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e6cb7-f002-47c7-9aff-d2ff5c98752b service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:26.517694 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e6cb7-f002-47c7-9aff-d2ff5c98752b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:26.526678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e6cb7-f002-47c7-9aff-d2ff5c98752b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:26.528120 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e6cb7-f002-47c7-9aff-d2ff5c98752b service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:26.536601 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1308/2570] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:36.440444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=20 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:36.532759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:47.551417 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d601d72-f808-4e55-98b1-06f74b37dec2 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:47.611862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d601d72-f808-4e55-98b1-06f74b37dec2 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:47.616699 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:47.617298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:47.617569 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:47.617834 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] PATH_INFO: `/v3/users/9da8a2a4b5c34857a75abbb2c255060f` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:47.618924 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9da8a2a4b5c34857a75abbb2c255060f)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:47.620901 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:47.665165 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9da8a2a4b5c34857a75abbb2c255060f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:47.665810 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9da8a2a4b5c34857a75abbb2c255060f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:47.672659 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9da8a2a4b5c34857a75abbb2c255060f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:47.673281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9da8a2a4b5c34857a75abbb2c255060f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:47.685941 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9da8a2a4b5c34857a75abbb2c255060f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:47.699017 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1263/2571] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:47 2022] DELETE /identity/v3/users/9da8a2a4b5c34857a75abbb2c255060f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:47.703470 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ff962-7918-42f9-9584-7bb80fb7d0e7 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:47.704007 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ff962-7918-42f9-9584-7bb80fb7d0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:47.704118 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ff962-7918-42f9-9584-7bb80fb7d0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:47.722733 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-049ff962-7918-42f9-9584-7bb80fb7d0e7 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:47.723953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-049ff962-7918-42f9-9584-7bb80fb7d0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:47.780547 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1309/2572] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:47.791268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:47.792702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:47.853130 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d32f0-96e6-4233-9659-d099e050cd01 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:47.856820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147d850-61cd-433f-85bc-a7591e5f5751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:47.857422 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147d850-61cd-433f-85bc-a7591e5f5751 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:47.857697 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147d850-61cd-433f-85bc-a7591e5f5751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:47.857966 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147d850-61cd-433f-85bc-a7591e5f5751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:47.867444 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147d850-61cd-433f-85bc-a7591e5f5751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:47.868897 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147d850-61cd-433f-85bc-a7591e5f5751 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:47.878177 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1264/2573] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:48.153904 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ff962-7918-42f9-9584-7bb80fb7d0e7 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:48.155329 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-049ff962-7918-42f9-9584-7bb80fb7d0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:48.169376 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ff962-7918-42f9-9584-7bb80fb7d0e7 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:48.174038 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:48.174736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.175094 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.175427 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] PATH_INFO: `/v3/projects/4413fbf30c154e7680c5c724eef354b1` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.176600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4413fbf30c154e7680c5c724eef354b1)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:48.178774 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:48.226141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4413fbf30c154e7680c5c724eef354b1', 'request_id': 'req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:48.231570 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4413fbf30c154e7680c5c724eef354b1', 'request_id': 'req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:48.239341 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4413fbf30c154e7680c5c724eef354b1', 'request_id': 'req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:48.251348 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] The token cache is being invalidate because project 4413fbf30c154e7680c5c724eef354b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:48.251633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:48.252295 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-dfdeaf49-dba6-4f48-8397-a7f01e3e5674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:48.253776 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1310/2574] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:48 2022] DELETE /identity/v3/projects/4413fbf30c154e7680c5c724eef354b1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:48.258894 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7553348-9a87-453e-a1b5-031e85d2477b None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.259180 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7553348-9a87-453e-a1b5-031e85d2477b None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.259452 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7553348-9a87-453e-a1b5-031e85d2477b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.279332 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-a7553348-9a87-453e-a1b5-031e85d2477b None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:48.280567 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a7553348-9a87-453e-a1b5-031e85d2477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:48.297862 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:48.340405 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1265/2575] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:48.348245 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e3fb5-5e28-41fd-bb4b-1b42d12a26c4 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.348529 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e3fb5-5e28-41fd-bb4b-1b42d12a26c4 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.348853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e3fb5-5e28-41fd-bb4b-1b42d12a26c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.369093 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-e52e3fb5-5e28-41fd-bb4b-1b42d12a26c4 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:48.370294 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-e52e3fb5-5e28-41fd-bb4b-1b42d12a26c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:48.378106 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1311/2576] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:48.382445 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7553348-9a87-453e-a1b5-031e85d2477b None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:48.392254 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7553348-9a87-453e-a1b5-031e85d2477b None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:48.396903 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58eb80-6918-45a0-9e31-e71c90ea62f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:48.397503 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58eb80-6918-45a0-9e31-e71c90ea62f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.398402 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58eb80-6918-45a0-9e31-e71c90ea62f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.398898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58eb80-6918-45a0-9e31-e71c90ea62f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.399274 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58eb80-6918-45a0-9e31-e71c90ea62f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 18:06:48.399575 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58eb80-6918-45a0-9e31-e71c90ea62f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:48.401208 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58eb80-6918-45a0-9e31-e71c90ea62f9 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:48.412545 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1266/2577] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Oct 14 18:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 18:06:48.417683 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e3fb5-5e28-41fd-bb4b-1b42d12a26c4 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:48.426903 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e3fb5-5e28-41fd-bb4b-1b42d12a26c4 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:48.432357 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b3b91-f58f-423c-a03e-cbedd051a6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:48.432941 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b3b91-f58f-423c-a03e-cbedd051a6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.433217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b3b91-f58f-423c-a03e-cbedd051a6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.433489 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b3b91-f58f-423c-a03e-cbedd051a6b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.434115 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b3b91-f58f-423c-a03e-cbedd051a6b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:48.436140 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b3b91-f58f-423c-a03e-cbedd051a6b0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:48.463479 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1312/2578] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:48 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:48.467561 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58eb80-6918-45a0-9e31-e71c90ea62f9 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:48.478014 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58eb80-6918-45a0-9e31-e71c90ea62f9 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:48.532885 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d03dc-8c5c-4b18-8b2c-3377bd9019f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:48.533509 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d03dc-8c5c-4b18-8b2c-3377bd9019f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.533782 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d03dc-8c5c-4b18-8b2c-3377bd9019f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.534048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d03dc-8c5c-4b18-8b2c-3377bd9019f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.534664 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d03dc-8c5c-4b18-8b2c-3377bd9019f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:48.536589 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d03dc-8c5c-4b18-8b2c-3377bd9019f3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:48.558779 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1267/2579] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:48 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:48.562874 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b3b91-f58f-423c-a03e-cbedd051a6b0 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:48.573975 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b3b91-f58f-423c-a03e-cbedd051a6b0 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:48.578528 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417db564-a319-4578-a9de-7c527a15a7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:48.579114 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417db564-a319-4578-a9de-7c527a15a7dc admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.579387 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417db564-a319-4578-a9de-7c527a15a7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.579655 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417db564-a319-4578-a9de-7c527a15a7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.580380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417db564-a319-4578-a9de-7c527a15a7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:48.581966 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417db564-a319-4578-a9de-7c527a15a7dc admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:48.592307 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1313/2580] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:48.596048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d03dc-8c5c-4b18-8b2c-3377bd9019f3 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:48.606418 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d03dc-8c5c-4b18-8b2c-3377bd9019f3 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:48.611692 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd9f97-94a8-4317-a51d-04385e218286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:48.612353 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd9f97-94a8-4317-a51d-04385e218286 admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.612631 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd9f97-94a8-4317-a51d-04385e218286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.612922 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd9f97-94a8-4317-a51d-04385e218286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.613583 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd9f97-94a8-4317-a51d-04385e218286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:48.615204 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd9f97-94a8-4317-a51d-04385e218286 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:48.626222 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1268/2581] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:48.630273 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417db564-a319-4578-a9de-7c527a15a7dc admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:48.640630 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417db564-a319-4578-a9de-7c527a15a7dc admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:48.645161 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c2999-20c1-4a32-9209-9032275a423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:48.645790 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c2999-20c1-4a32-9209-9032275a423c admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.646064 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c2999-20c1-4a32-9209-9032275a423c admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.646339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c2999-20c1-4a32-9209-9032275a423c admin admin] PATH_INFO: `/v3/projects/ce34ce921fbd42f3bd0255599c39c0c6/users/13cc81b76c054ae1a041abdd5900442f/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.661085 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c2999-20c1-4a32-9209-9032275a423c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce34ce921fbd42f3bd0255599c39c0c6, user_id=13cc81b76c054ae1a041abdd5900442f, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:48.663789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c2999-20c1-4a32-9209-9032275a423c admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:48.676235 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1314/2582] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:48 2022] PUT /identity/v3/projects/ce34ce921fbd42f3bd0255599c39c0c6/users/13cc81b76c054ae1a041abdd5900442f/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:48.687114 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e491e3-c973-4f8b-b770-41f18596f68d None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.687421 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e491e3-c973-4f8b-b770-41f18596f68d None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.687694 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e491e3-c973-4f8b-b770-41f18596f68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.716898 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-91e491e3-c973-4f8b-b770-41f18596f68d None None] MFA Rules not processed for user `13cc81b76c054ae1a041abdd5900442f`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:48.718112 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-91e491e3-c973-4f8b-b770-41f18596f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:48.780592 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1269/2583] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:48.785105 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c2999-20c1-4a32-9209-9032275a423c admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:48.806923 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:48.850498 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c2999-20c1-4a32-9209-9032275a423c admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:48.855020 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df6ad1-daef-49e5-aba6-2ca88265e388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:48.855605 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df6ad1-daef-49e5-aba6-2ca88265e388 admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.855889 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df6ad1-daef-49e5-aba6-2ca88265e388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.856167 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df6ad1-daef-49e5-aba6-2ca88265e388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.856814 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df6ad1-daef-49e5-aba6-2ca88265e388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:48.858690 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df6ad1-daef-49e5-aba6-2ca88265e388 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:48.885174 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1315/2584] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:48 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:48.889387 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e491e3-c973-4f8b-b770-41f18596f68d None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:48.900011 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e491e3-c973-4f8b-b770-41f18596f68d None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:48.912022 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:48.956478 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e970bd0-c5f4-49d5-8896-cac467f115fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:48.957080 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e970bd0-c5f4-49d5-8896-cac467f115fc admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:48.957356 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e970bd0-c5f4-49d5-8896-cac467f115fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:48.957621 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e970bd0-c5f4-49d5-8896-cac467f115fc admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:48.958233 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e970bd0-c5f4-49d5-8896-cac467f115fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:48.960188 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e970bd0-c5f4-49d5-8896-cac467f115fc admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:48.981217 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1270/2585] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:48 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:48.985288 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df6ad1-daef-49e5-aba6-2ca88265e388 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:48.996196 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df6ad1-daef-49e5-aba6-2ca88265e388 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:49.000669 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b479b-24f6-4e70-a1cc-13b3faca7edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:49.001251 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b479b-24f6-4e70-a1cc-13b3faca7edf admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.001524 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b479b-24f6-4e70-a1cc-13b3faca7edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.001789 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b479b-24f6-4e70-a1cc-13b3faca7edf admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.002420 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b479b-24f6-4e70-a1cc-13b3faca7edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:49.004048 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b479b-24f6-4e70-a1cc-13b3faca7edf admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:49.014765 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1316/2586] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:49.018629 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e970bd0-c5f4-49d5-8896-cac467f115fc admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:49.028617 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e970bd0-c5f4-49d5-8896-cac467f115fc admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:49.033207 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb52b2-df35-4024-86cb-28aa8b93394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:49.033786 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb52b2-df35-4024-86cb-28aa8b93394c admin admin] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.034063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb52b2-df35-4024-86cb-28aa8b93394c admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.034331 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb52b2-df35-4024-86cb-28aa8b93394c admin admin] PATH_INFO: `/v3/roles` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.034958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb52b2-df35-4024-86cb-28aa8b93394c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:49.036572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb52b2-df35-4024-86cb-28aa8b93394c admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:49.046281 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1271/2587] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:49.050270 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b479b-24f6-4e70-a1cc-13b3faca7edf admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:49.061022 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b479b-24f6-4e70-a1cc-13b3faca7edf admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:49.065572 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7d395-e5c8-4bb3-8eca-64699a674b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:49.066206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7d395-e5c8-4bb3-8eca-64699a674b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.066509 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7d395-e5c8-4bb3-8eca-64699a674b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.066777 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7d395-e5c8-4bb3-8eca-64699a674b32 admin admin] PATH_INFO: `/v3/projects/a205d047a2514501a8bb52346b576ae3/users/7c3c6ac24d3c42ae8321f3f1aee45e02/roles/9ab8d70da5894a2f99c6d22e5351baa3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.082464 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7d395-e5c8-4bb3-8eca-64699a674b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a205d047a2514501a8bb52346b576ae3, user_id=7c3c6ac24d3c42ae8321f3f1aee45e02, role_id=9ab8d70da5894a2f99c6d22e5351baa3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:49.085229 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7d395-e5c8-4bb3-8eca-64699a674b32 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:49.097963 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1317/2588] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:49 2022] PUT /identity/v3/projects/a205d047a2514501a8bb52346b576ae3/users/7c3c6ac24d3c42ae8321f3f1aee45e02/roles/9ab8d70da5894a2f99c6d22e5351baa3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:49.109218 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2bff6-a746-4996-9bab-a679870a1d16 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.109432 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2bff6-a746-4996-9bab-a679870a1d16 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.109702 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2bff6-a746-4996-9bab-a679870a1d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.140558 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-d9c2bff6-a746-4996-9bab-a679870a1d16 None None] MFA Rules not processed for user `7c3c6ac24d3c42ae8321f3f1aee45e02`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:49.141791 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-d9c2bff6-a746-4996-9bab-a679870a1d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:49.208700 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1272/2589] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:49.213577 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7d395-e5c8-4bb3-8eca-64699a674b32 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:49.274177 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7d395-e5c8-4bb3-8eca-64699a674b32 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:49.278700 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2a64d-5172-4a33-9c13-ff233e9b61cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:49.279285 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a64d-5172-4a33-9c13-ff233e9b61cc admin admin] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.279555 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a64d-5172-4a33-9c13-ff233e9b61cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.279819 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a64d-5172-4a33-9c13-ff233e9b61cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.280456 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2a64d-5172-4a33-9c13-ff233e9b61cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:49.282334 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2a64d-5172-4a33-9c13-ff233e9b61cc admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:49.308935 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1318/2590] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Oct 14 18:06:49 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:49.312967 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2bff6-a746-4996-9bab-a679870a1d16 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:49.323319 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2bff6-a746-4996-9bab-a679870a1d16 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:49.381115 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499b343-b4db-4282-bb40-c70d9cf753d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:49.381651 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499b343-b4db-4282-bb40-c70d9cf753d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.386134 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499b343-b4db-4282-bb40-c70d9cf753d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.386489 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499b343-b4db-4282-bb40-c70d9cf753d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.387183 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499b343-b4db-4282-bb40-c70d9cf753d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:49.389828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499b343-b4db-4282-bb40-c70d9cf753d3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:49.411860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1273/2591] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Oct 14 18:06:49 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 18:06:49.416598 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2a64d-5172-4a33-9c13-ff233e9b61cc admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:49.427380 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2a64d-5172-4a33-9c13-ff233e9b61cc admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:49.433088 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe42d1d-9dff-4fdb-b51a-c08819f85f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:49.433678 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe42d1d-9dff-4fdb-b51a-c08819f85f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.433953 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe42d1d-9dff-4fdb-b51a-c08819f85f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.434219 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe42d1d-9dff-4fdb-b51a-c08819f85f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.434853 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe42d1d-9dff-4fdb-b51a-c08819f85f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:49.436520 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe42d1d-9dff-4fdb-b51a-c08819f85f70 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:49.448188 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1319/2592] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:49.452298 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499b343-b4db-4282-bb40-c70d9cf753d3 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:49.462661 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499b343-b4db-4282-bb40-c70d9cf753d3 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:49.467315 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f9036-2f3a-4e24-b347-19c443c815ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:49.468007 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9036-2f3a-4e24-b347-19c443c815ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.468273 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9036-2f3a-4e24-b347-19c443c815ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.468537 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9036-2f3a-4e24-b347-19c443c815ef admin admin] PATH_INFO: `/v3/projects/adf669e786fa43e08fd47588c0034cf0/users/c278c700a29d43c6b28d8412e031e3d3/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.481429 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:49.484996 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f9036-2f3a-4e24-b347-19c443c815ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf669e786fa43e08fd47588c0034cf0, user_id=c278c700a29d43c6b28d8412e031e3d3, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:49.488612 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f9036-2f3a-4e24-b347-19c443c815ef admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:49.502063 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1274/2593] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:49 2022] PUT /identity/v3/projects/adf669e786fa43e08fd47588c0034cf0/users/c278c700a29d43c6b28d8412e031e3d3/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:49.506002 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe42d1d-9dff-4fdb-b51a-c08819f85f70 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:49.515745 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe42d1d-9dff-4fdb-b51a-c08819f85f70 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:49.556767 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:49.572759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c82dd3-ee82-4681-9d0d-e654cd6bae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:49.573360 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c82dd3-ee82-4681-9d0d-e654cd6bae96 admin admin] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.573638 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c82dd3-ee82-4681-9d0d-e654cd6bae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.573911 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c82dd3-ee82-4681-9d0d-e654cd6bae96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.574539 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c82dd3-ee82-4681-9d0d-e654cd6bae96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:49.576199 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c82dd3-ee82-4681-9d0d-e654cd6bae96 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:49.586279 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1320/2594] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Oct 14 18:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 18:06:49.590513 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f9036-2f3a-4e24-b347-19c443c815ef admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:49.601205 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f9036-2f3a-4e24-b347-19c443c815ef admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:49.605896 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f716c5-1b7a-419c-a18c-c798d0c2a833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:49.606534 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f716c5-1b7a-419c-a18c-c798d0c2a833 admin admin] REQUEST_METHOD: `PUT` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.606812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f716c5-1b7a-419c-a18c-c798d0c2a833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.607110 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f716c5-1b7a-419c-a18c-c798d0c2a833 admin admin] PATH_INFO: `/v3/projects/adf669e786fa43e08fd47588c0034cf0/users/c278c700a29d43c6b28d8412e031e3d3/roles/45b67ba9922c4b16baae3930d3de1b6c` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.609053 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f716c5-1b7a-419c-a18c-c798d0c2a833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf669e786fa43e08fd47588c0034cf0, user_id=c278c700a29d43c6b28d8412e031e3d3, role_id=45b67ba9922c4b16baae3930d3de1b6c)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:49.611649 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f716c5-1b7a-419c-a18c-c798d0c2a833 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:49.621911 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1275/2595] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Oct 14 18:06:49 2022] PUT /identity/v3/projects/adf669e786fa43e08fd47588c0034cf0/users/c278c700a29d43c6b28d8412e031e3d3/roles/45b67ba9922c4b16baae3930d3de1b6c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:49.632165 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b53887-8024-4199-bc0a-df7c20db4161 None None] REQUEST_METHOD: `POST` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.632391 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b53887-8024-4199-bc0a-df7c20db4161 None None] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.632677 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b53887-8024-4199-bc0a-df7c20db4161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.661659 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.auth.core [None req-08b53887-8024-4199-bc0a-df7c20db4161 None None] MFA Rules not processed for user `c278c700a29d43c6b28d8412e031e3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:49.662859 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-08b53887-8024-4199-bc0a-df7c20db4161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:49.732434 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1321/2596] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:49.743900 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f716c5-1b7a-419c-a18c-c798d0c2a833 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:49.745611 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-18f716c5-1b7a-419c-a18c-c798d0c2a833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:49.802059 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f716c5-1b7a-419c-a18c-c798d0c2a833 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:49.805980 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5874d60-3375-4a22-bafc-2b613337e76a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:49.806573 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5874d60-3375-4a22-bafc-2b613337e76a service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:49.806846 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5874d60-3375-4a22-bafc-2b613337e76a service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:49.807115 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5874d60-3375-4a22-bafc-2b613337e76a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:49.862818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5874d60-3375-4a22-bafc-2b613337e76a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:49.864268 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5874d60-3375-4a22-bafc-2b613337e76a service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:49.873216 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1276/2597] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:51.386585 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b53887-8024-4199-bc0a-df7c20db4161 None None] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:51.388039 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-08b53887-8024-4199-bc0a-df7c20db4161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:51.400385 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b53887-8024-4199-bc0a-df7c20db4161 None None] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:51.403846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0500c-d9ea-4913-817d-226e0ddc0c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:51.404486 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0500c-d9ea-4913-817d-226e0ddc0c4a service cinder] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:51.404755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0500c-d9ea-4913-817d-226e0ddc0c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:51.405017 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0500c-d9ea-4913-817d-226e0ddc0c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:51.459303 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0500c-d9ea-4913-817d-226e0ddc0c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:51.461176 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0500c-d9ea-4913-817d-226e0ddc0c4a service cinder] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:51.469736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1322/2598] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:52.006743 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5874d60-3375-4a22-bafc-2b613337e76a service cinder] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:52.015138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5874d60-3375-4a22-bafc-2b613337e76a service cinder] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:52.018663 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f08677-c47b-4245-9b27-b995ed5f04c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f51ee9e45fb4dda8d5b9740aa0ac828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:52.019248 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f08677-c47b-4245-9b27-b995ed5f04c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:52.019520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f08677-c47b-4245-9b27-b995ed5f04c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:52.019804 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f08677-c47b-4245-9b27-b995ed5f04c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:52.029036 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f08677-c47b-4245-9b27-b995ed5f04c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:52.030430 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f08677-c47b-4245-9b27-b995ed5f04c0 service cinder] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:52.040242 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1277/2599] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:55.927829 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0500c-d9ea-4913-817d-226e0ddc0c4a service cinder] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:55.992948 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0500c-d9ea-4913-817d-226e0ddc0c4a service cinder] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:55.997607 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:55.998234 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:55.998511 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:55.998781 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] PATH_INFO: `/v3/users/13cc81b76c054ae1a041abdd5900442f` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:55.999913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13cc81b76c054ae1a041abdd5900442f)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:56.001844 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:56.007386 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=100 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:56.054053 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13cc81b76c054ae1a041abdd5900442f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.054712 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13cc81b76c054ae1a041abdd5900442f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.063256 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13cc81b76c054ae1a041abdd5900442f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.063949 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13cc81b76c054ae1a041abdd5900442f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.076791 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13cc81b76c054ae1a041abdd5900442f'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.091312 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1323/2600] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:55 2022] DELETE /identity/v3/users/13cc81b76c054ae1a041abdd5900442f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:56.096213 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0596c3-edbc-4359-affc-7e8b6891b6f1 None None] REQUEST_METHOD: `POST` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:56.096723 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0596c3-edbc-4359-affc-7e8b6891b6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:56.096891 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0596c3-edbc-4359-affc-7e8b6891b6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:56.116196 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.auth.core [None req-7d0596c3-edbc-4359-affc-7e8b6891b6f1 None None] MFA Rules not processed for user `92acc9f005ce4ccfaadbc688dbb479a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 18:06:56.117415 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7d0596c3-edbc-4359-affc-7e8b6891b6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:56.180800 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1278/2601] 10.222.0.27 () {58 vars in 958 bytes} [Fri Oct 14 18:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 18:06:56.190544 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:56.191988 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:56.251868 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee4f43-bc79-4430-b885-a3624412c617 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:56.256873 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fe33a-d8ff-4654-b447-24b95f8002a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:56.257633 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fe33a-d8ff-4654-b447-24b95f8002a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:56.257910 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fe33a-d8ff-4654-b447-24b95f8002a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:56.258226 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fe33a-d8ff-4654-b447-24b95f8002a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:56.267701 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fe33a-d8ff-4654-b447-24b95f8002a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:56.269142 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fe33a-d8ff-4654-b447-24b95f8002a3 service neutron] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:56.279695 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1324/2602] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:06:56.543077 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0596c3-edbc-4359-affc-7e8b6891b6f1 None None] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:56.544587 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-7d0596c3-edbc-4359-affc-7e8b6891b6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:56.558640 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0596c3-edbc-4359-affc-7e8b6891b6f1 None None] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:56.569704 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:56.570204 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:56.570457 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:56.570737 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] PATH_INFO: `/v3/projects/ce34ce921fbd42f3bd0255599c39c0c6` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:56.572006 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce34ce921fbd42f3bd0255599c39c0c6)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:56.574121 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:56.590403 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:56.622714 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce34ce921fbd42f3bd0255599c39c0c6', 'request_id': 'req-8a3ff15e-c169-4327-8302-378f7e7bad7f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.628107 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce34ce921fbd42f3bd0255599c39c0c6', 'request_id': 'req-8a3ff15e-c169-4327-8302-378f7e7bad7f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.636423 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce34ce921fbd42f3bd0255599c39c0c6', 'request_id': 'req-8a3ff15e-c169-4327-8302-378f7e7bad7f'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.648288 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] The token cache is being invalidate because project ce34ce921fbd42f3bd0255599c39c0c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:56.648572 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.649204 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.650690 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1279/2603] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:56 2022] DELETE /identity/v3/projects/ce34ce921fbd42f3bd0255599c39c0c6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:56.654699 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fe33a-d8ff-4654-b447-24b95f8002a3 service neutron] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:56.664778 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fe33a-d8ff-4654-b447-24b95f8002a3 service neutron] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:56.721153 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:56.721762 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:56.722032 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:56.722297 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] PATH_INFO: `/v3/users/7c3c6ac24d3c42ae8321f3f1aee45e02` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:56.723409 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c3c6ac24d3c42ae8321f3f1aee45e02)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:56.725989 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:56.755571 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:56.774755 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c3c6ac24d3c42ae8321f3f1aee45e02'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.775403 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c3c6ac24d3c42ae8321f3f1aee45e02'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.782141 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c3c6ac24d3c42ae8321f3f1aee45e02'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.782759 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c3c6ac24d3c42ae8321f3f1aee45e02'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.795389 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c3c6ac24d3c42ae8321f3f1aee45e02'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:56.808735 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1325/2604] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:56 2022] DELETE /identity/v3/users/7c3c6ac24d3c42ae8321f3f1aee45e02 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:57.075362 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:57.076847 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:57.090759 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ff15e-c169-4327-8302-378f7e7bad7f admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:57.147818 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:57.148520 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:57.148758 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:57.149029 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] PATH_INFO: `/v3/projects/a205d047a2514501a8bb52346b576ae3` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:57.150135 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a205d047a2514501a8bb52346b576ae3)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:57.152093 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:57.198767 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a205d047a2514501a8bb52346b576ae3', 'request_id': 'req-a7a13d55-ef7d-4d8a-b276-105c14e736fc'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.204303 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a205d047a2514501a8bb52346b576ae3', 'request_id': 'req-a7a13d55-ef7d-4d8a-b276-105c14e736fc'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.212225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a205d047a2514501a8bb52346b576ae3', 'request_id': 'req-a7a13d55-ef7d-4d8a-b276-105c14e736fc'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.223517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] The token cache is being invalidate because project a205d047a2514501a8bb52346b576ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:57.223807 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.224841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.226338 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1280/2605] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:57 2022] DELETE /identity/v3/projects/a205d047a2514501a8bb52346b576ae3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:57.230441 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:57.231845 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:57.245217 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ef925-dadf-4b44-8e30-5ad6209488a3 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:57.300736 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:57.301356 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:57.301626 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:57.302656 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] PATH_INFO: `/v3/users/c278c700a29d43c6b28d8412e031e3d3` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:57.304249 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c278c700a29d43c6b28d8412e031e3d3)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:57.306146 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:57.353206 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c278c700a29d43c6b28d8412e031e3d3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.353855 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c278c700a29d43c6b28d8412e031e3d3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.360714 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c278c700a29d43c6b28d8412e031e3d3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.361324 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c278c700a29d43c6b28d8412e031e3d3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.373709 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c278c700a29d43c6b28d8412e031e3d3'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.386996 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1326/2606] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:06:57 2022] DELETE /identity/v3/users/c278c700a29d43c6b28d8412e031e3d3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:06:57.658026 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:06:57.660381 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:06:57.678199 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a13d55-ef7d-4d8a-b276-105c14e736fc admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:06:57.715859 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:06:57.743089 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:06:57.743645 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:06:57.743948 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:06:57.744219 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] PATH_INFO: `/v3/projects/adf669e786fa43e08fd47588c0034cf0` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:06:57.745312 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf669e786fa43e08fd47588c0034cf0)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:06:57.747220 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:06:57.793812 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf669e786fa43e08fd47588c0034cf0', 'request_id': 'req-00d8f822-905d-48fc-a796-ed1964cd7023'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.799474 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf669e786fa43e08fd47588c0034cf0', 'request_id': 'req-00d8f822-905d-48fc-a796-ed1964cd7023'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.807955 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf669e786fa43e08fd47588c0034cf0', 'request_id': 'req-00d8f822-905d-48fc-a796-ed1964cd7023'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.819805 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] The token cache is being invalidate because project adf669e786fa43e08fd47588c0034cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:06:57.820125 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.820761 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:06:57.822250 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1281/2607] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:06:57 2022] DELETE /identity/v3/projects/adf669e786fa43e08fd47588c0034cf0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:07:07.383371 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=72,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:07:07.816225 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:07:36.120664 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:07:36.122301 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:07:36.209078 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ae030-5076-47b1-bb73-c333bd62e4b0 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:07:36.275715 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96217c03-2814-42ae-95ed-544f973f0232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42968016de8470c933655e921fb3394', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:07:36.276367 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96217c03-2814-42ae-95ed-544f973f0232 service nova] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:07:36.276637 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96217c03-2814-42ae-95ed-544f973f0232 service nova] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:07:36.276905 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96217c03-2814-42ae-95ed-544f973f0232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:07:36.278600 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-96217c03-2814-42ae-95ed-544f973f0232 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:07:36.291483 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96217c03-2814-42ae-95ed-544f973f0232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:07:36.293083 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96217c03-2814-42ae-95ed-544f973f0232 service nova] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:07:36.304105 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1327/2608] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:07:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:07:46.299668 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=62 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:07:46.414504 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:07:46.416048 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:07:46.472957 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f822-905d-48fc-a796-ed1964cd7023 admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:07:46.476653 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4ebfc-22a9-42ab-b146-df06bb540858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c17f274e4c14f2f99650844642437a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:07:46.477253 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4ebfc-22a9-42ab-b146-df06bb540858 service neutron] REQUEST_METHOD: `GET` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:07:46.477530 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4ebfc-22a9-42ab-b146-df06bb540858 service neutron] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:07:46.477798 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4ebfc-22a9-42ab-b146-df06bb540858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:07:46.479515 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-fbf4ebfc-22a9-42ab-b146-df06bb540858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:07:46.542440 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4ebfc-22a9-42ab-b146-df06bb540858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:07:46.543864 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4ebfc-22a9-42ab-b146-df06bb540858 service neutron] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:07:46.554609 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1282/2609] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:07:52.676804 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96217c03-2814-42ae-95ed-544f973f0232 service nova] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:07:52.678846 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-96217c03-2814-42ae-95ed-544f973f0232 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:07:52.694417 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96217c03-2814-42ae-95ed-544f973f0232 service nova] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:07:52.699550 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:07:52.700227 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:07:52.700532 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:07:52.700806 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] PATH_INFO: `/v3/users/072cedf1a6a24f6fa7d7ed92981344dd` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:07:52.701980 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=072cedf1a6a24f6fa7d7ed92981344dd)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:07:52.704632 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:07:52.752015 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '072cedf1a6a24f6fa7d7ed92981344dd'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:52.752591 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '072cedf1a6a24f6fa7d7ed92981344dd'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:52.759499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '072cedf1a6a24f6fa7d7ed92981344dd'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:52.760171 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '072cedf1a6a24f6fa7d7ed92981344dd'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:52.772180 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '072cedf1a6a24f6fa7d7ed92981344dd'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:52.785457 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1328/2610] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:07:52 2022] DELETE /identity/v3/users/072cedf1a6a24f6fa7d7ed92981344dd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:07:52.958841 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4ebfc-22a9-42ab-b146-df06bb540858 service neutron] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:07:52.969290 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4ebfc-22a9-42ab-b146-df06bb540858 service neutron] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:07:53.026145 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:07:53.026701 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:07:53.026979 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:07:53.027246 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] PATH_INFO: `/v3/projects/87b79c338b0047dfb68e9a5b9b16c51b` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:07:53.028402 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b79c338b0047dfb68e9a5b9b16c51b)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:07:53.031043 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:07:53.076386 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b79c338b0047dfb68e9a5b9b16c51b', 'request_id': 'req-0c65358e-3356-4fad-87a5-0ef2560371db'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.078836 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:07:53.083724 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b79c338b0047dfb68e9a5b9b16c51b', 'request_id': 'req-0c65358e-3356-4fad-87a5-0ef2560371db'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.090802 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b79c338b0047dfb68e9a5b9b16c51b', 'request_id': 'req-0c65358e-3356-4fad-87a5-0ef2560371db'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.102024 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] The token cache is being invalidate because project 87b79c338b0047dfb68e9a5b9b16c51b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:07:53.102306 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.102957 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.104510 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1283/2611] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:07:52 2022] DELETE /identity/v3/projects/87b79c338b0047dfb68e9a5b9b16c51b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:07:53.109019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:07:53.110407 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:07:53.123374 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c940157-8c54-4386-aad4-150dbe117a42 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:07:53.179408 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:07:53.180055 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:07:53.180333 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:07:53.180602 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] PATH_INFO: `/v3/users/ad5e8ab4050c4cf0969c0b09ef652544` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:07:53.181713 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5e8ab4050c4cf0969c0b09ef652544)` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:07:53.183612 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:07:53.228812 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5e8ab4050c4cf0969c0b09ef652544'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.229446 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5e8ab4050c4cf0969c0b09ef652544'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.237230 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5e8ab4050c4cf0969c0b09ef652544'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.237984 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5e8ab4050c4cf0969c0b09ef652544'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.251368 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.notifications [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5e8ab4050c4cf0969c0b09ef652544'} {{(pid=81267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.253019 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:07:53.266913 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1329/2612] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Oct 14 18:07:53 2022] DELETE /identity/v3/users/ad5e8ab4050c4cf0969c0b09ef652544 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:07:53.449003 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] Authenticating user token {{(pid=81266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:07:53.450466 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.fernet_utils [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:07:53.463958 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65358e-3356-4fad-87a5-0ef2560371db admin admin] Validating token access rules against request {{(pid=81266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:07:53.520517 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92acc9f005ce4ccfaadbc688dbb479a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a41cbe9ae3574988a4c391824f7f340a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:07:53.521189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:07:53.521447 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:07:53.521716 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] PATH_INFO: `/v3/projects/97ffbaca123c421f89cfb397545fbe66` {{(pid=81266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:07:53.522820 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ffbaca123c421f89cfb397545fbe66)` {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:07:53.524764 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] RBAC: Authorization granted {{(pid=81266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:07:53.571406 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ffbaca123c421f89cfb397545fbe66', 'request_id': 'req-790f684e-abb4-48c8-9381-a6e2750509a3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.576860 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ffbaca123c421f89cfb397545fbe66', 'request_id': 'req-790f684e-abb4-48c8-9381-a6e2750509a3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.584138 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ffbaca123c421f89cfb397545fbe66', 'request_id': 'req-790f684e-abb4-48c8-9381-a6e2750509a3'} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.595821 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] The token cache is being invalidate because project 97ffbaca123c421f89cfb397545fbe66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 18:07:53.596132 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.596828 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG keystone.notifications [None req-790f684e-abb4-48c8-9381-a6e2750509a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 18:07:53.598363 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: [pid: 81266|app: 0|req: 1284/2613] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Oct 14 18:07:53 2022] DELETE /identity/v3/projects/97ffbaca123c421f89cfb397545fbe66 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 18:08:03.263137 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:08:03.592189 dv-cinder-749155-15-iscsi devstack@keystone.service[81266]: DEBUG dbcounter [-] [81266] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 18:08:15.561499 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] Authenticating user token {{(pid=81267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 18:08:15.562961 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:08:15.630359 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a14f26-d63e-41de-bc7e-346acd7d9672 admin admin] Validating token access rules against request {{(pid=81267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 18:08:15.688461 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b20f87-6e94-49f2-ac10-bc51a1cf44fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73089bc9f85a405b86c4cd3f8cfa830a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009e40c8d9ec4f828b2ee0f735842aa0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 18:08:15.689063 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b20f87-6e94-49f2-ac10-bc51a1cf44fd service placement] REQUEST_METHOD: `GET` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 18:08:15.689339 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b20f87-6e94-49f2-ac10-bc51a1cf44fd service placement] SCRIPT_NAME: `/identity` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 18:08:15.689604 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b20f87-6e94-49f2-ac10-bc51a1cf44fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 18:08:15.691314 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.fernet_utils [None req-f3b20f87-6e94-49f2-ac10-bc51a1cf44fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 18:08:15.702621 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b20f87-6e94-49f2-ac10-bc51a1cf44fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 18:08:15.704233 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b20f87-6e94-49f2-ac10-bc51a1cf44fd service placement] RBAC: Authorization granted {{(pid=81267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 18:08:15.714042 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: [pid: 81267|app: 0|req: 1330/2614] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Oct 14 18:08:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 18:08:25.709822 dv-cinder-749155-15-iscsi devstack@keystone.service[81267]: DEBUG dbcounter [-] [81267] Writing DB stats keystone:SELECT=58 {{(pid=81267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}